Diversifying Parallel Ergodic Search: A Signature Kernel Evolution Strategy
Authors: Sreevardhan Sirigiri
Christian Hughes
Ian Abraham
Fabio Ramos
Venue: NeurIPS 2025 poster ()
Keywords: robotics exploration
trajectory optimization
ergodic search
signature transform
Primary area: reinforcement_learning
TL;DR: We introduce two complementary enhancements to Stein Variational Ergodic Search, yielding richer, more efficient exploration trajectories across diverse robotic benchmarks.

Abstract: Effective robotic exploration in continuous domains requires planning trajectories that maximize coverage over a predefined region. A recent development, Stein Variational Ergodic Search (SVES), proposed parallel ergodic exploration (a key approach within the field of robotic exploration), via Stein variational inference that computes a set of candidate trajectories approximating the posterior distribution over the solution space trajectories. While this approach leverages GPU parallelism well, the trajectories in the set might not be distinct enough, leading to a suboptimal set. In this paper, we propose two key methods to diversify the solution set of this approach.
First, we leverage the signature kernel within the SVES framework, introducing a pathwise, sequence-sensitive interaction that preserves the Markovian structure of the trajectories and naturally spreads paths across distinct regions of the search space. Second, we propose a derivative-free evolution-strategy interpretation of SVES that exploits batched, GPU-friendly fitness evaluations and can be paired with approximate gradients whenever analytic gradients of the kernel are unavailable or computationally intractable. The resulting method both retains SVES’s advantages while diversifying the solution set and extending its reach to black-box objectives. Across planar forest search, 3D quadrotor coverage, and model-predictive control benchmarks, our approach consistently reduces ergodic cost and produces markedly richer trajectory sets than SVES without significant extra tuning effort.

Diversifying Parallel Ergodic Search: A Signature Kernel Evolution Strategy
Sreevardhan Sirigiri The University of Sydney, Australia ssir49190uni sydney... edu. au
Christian Hughes
Yale University, USA christian. paristian.hughes@eeee. www.hhuyalll...ooomomoo edu
lan Abraham
Fabio Ramos
Yale University, USA ian. abraham@yale, edu
Nvidia, USA
The University of Sydney, Australia
fabio ramos@sydney edu.
Abstract
Effective robotic exploration in continuous domains requires planning trajectories that maximize coverage over a predefined region. A recent development, Stein Variational Ergodic Search (SVES), proposed paralfel ergodic exploration (a key approach within the field of robotic exploration), via Stein variational inference that computes of candidate trajectories approximating the posterior distribution over the solution space trajectories. While this approach leverages GPU parallelism well, the trajectories in the set might not distinct enough, leading suboptimal set. In this paper, we propose two key methods to diversify the solution set of this approach. First, we leverage the signature kernel within the SVES framework, introducing a pathwise, sequence-sensitive interaction that preserves the Markovian structure of the trajectories and naturally spreads paths across distinct regions of the search space. Second, we propose a derivative-free evolution evolution-strategy interpretation of SVES that exploits batched, GPU-friendly fitness evaluations and can be paired with approximate gradients whenever analytic gradients of the kernel are unavailable or computationally intractable. The resulting method both retains SVES's advantages while diversifying the solution set and extending its reach to black-box objectives. Across planar forest search, 3D quadrotor coverage, and model-predicee control benchmarks, approach consistently reduces ergodic cost and produces markedly richer trajectory sets than SVES without significant extra tuning effort.
1 Introduction
Robotic exploration-tle autonomous process by which robots survey unknown partially mapped environments collect information, build maps, and accomplish tasks such as bearch-and-resous environmental monitoring, infrastructure inspection. The success of robotic exploration hinders on the reliability of the robot to adapt its exploratory opteration in dynamic or unstructured settings where pre-programmed paths may fail. A key approach within this field is ergodic search [1, 2], in which trajectories are generated so that the robot's time-averaged visitation frequency matches a desired spatial distribution of information. By ensuring that time spent in each region is proportional to its importance, ergodic search provides systematic, efficient coverage of the domain and avoids both redundantly revisiting well-known areas and neglecting critical regions. In such applications where a robot must systematically explore an area, having multiple, robust, high-quality and diverse set of trajectories is critical to adapt dynamic environments and quickly switch between search
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

strategies. Moreover, set of diverse solutions aid in avoiding "bad" local minima and lead to a better optimization outcome.
Recent progress in euriosity- and information-basee exploration has enabled robots to survey large, unstructured environments [3-7]. However, most existing methods rely exclusively on information- maximizing strategy, which often leads to myopic behavior: agents greedily seek immediate gains in information without regard for long-term, strategically advantageous states.
Ergodic exploration tech- niques address this limita- tion by casting exploration coverage problem, op- timizing time-averaged tra- jectory statistics ensure sustained visitation of high- information regions [8, 9]. Concretely, these methods optimize the spatial distribu- tion of trajectory dwell time against an expected infor- mation measure, allowing robots to plan exploratory paths that effectively handle multi-modal search land- scapes [1]. Despite their demonstrated effectiveness, current ergodic algorithms still optimize only a single search policy at time, con- straining a robot's 's ability to adapt when tasks environments change.
(a) RBF kernel
(b) Signature kemel
Figure I: illustrates the qualitative impact of our signature- enhancement: where trajectories generated using the RBF kernel (panel a) not only cluster around the domain center but also produce rough, lower-quality paths, the signature kernel (panel b) yiclds smooth, high- quality trajectories that spread more uniformly across the entire domain, achieving significantly richer ergodic coverage. These trajectories were flown by Crazyflie drones in a real-world setting as part of the experiment described in Section 7.2.
Prior work has observed that the non-convex nature of the ergodic objective can yield multiple locally optimal trajectories under different initializations [9]. Yet explicitly reasoning over-an sampling from distribution of such trajectories computationally prohibitive. Moreover, there is no assurance that distinct initial conditions will avoid collapsing onto the same ergodic solution, a critical shortcoming in online exploration scenarios where mode collapse can incur catastrophic performance failures.
Stein variational inference methods show promise in providing the necessary tools to approximate distributions of trajectories in computationally tractable manner [10]. Motivated by these strengths, [2] proposes Stein variational ergodic search (SVES), a formulation of ergodic exploration variational inference problem: by applying Stein variational gradient descent to the space of robot trajectories. However, SVES neither fully exploits the inherent Markovian structure trajectories nor avoids a strong dependence on gradient information. Moreover, although SVES alludes to promoting diversity among trajectories, it does not concretely define this notion or provide what "diversity" entails.
Stein
Contributions: We address the challenges mentioned above by "diversifying" Stein Variational Ergodic Search (SVES). Our approach is built on two complementary pillars. First, we leverage the signature kernel similar [11], enabling us to encode the intrinsic sequential (or Markovian) and geometric properties of trajectories. The signature kernel naturally discriminates between subtly different paths, thus promoting diversity. Second, we incorporate Stein Variational Covariance Matrix Adaptation Evolution Strategy (SV-CMA-ES) [12] and Simultaneous Perturbation Stochastic Approximation (SPSA) [13] into SVES, drawing on recent progress in Stein Variational Evolution Strategies, this approach not only eliminates the need for gradients-ore -ofen costly to compute by harnessing GPU parallelism, making it viable for real-time applications, but also enables the use of complex non-differentiable or black-box loss functions and constraints. Later show empirically that this approach leads to better optimization results and more diverse set of solutions.
Paper organization: The remainder of the paper is organized as follows. In Section 2 we formalize the ergodic search problem, Section 3 reviews Stein variational ergodic search, Section introduces

---

the signature transform and signature kernel, Sections 5 and 6 present our diversification strategies using the signature kernel and SV-CMA-ES respectively, Section 7 reports empirical results, Section 8 concludes and discusses the limitations and best practices.
2
Problem Statement
We consider a robot with state space S R and control space U C R" whose continuous tame dynamics are
+(1) = [x((1,, u(O), x=0) za E Æx, a : [0. 7] +
robot's space trajectory æ<t) [0, 7] + X' is defined as the solution of the corresponding initial- value problem (integrated over time):
=tt) = z(0) + J. f((((), u(E)) de.
(I)
We denote the set of all such finite-horizon trajectories by Ur>o (10, ([, T), X), , as the set of all continuous trajectories defined on arbitrary finite time intervals. The robot explores a bounded domain S [0, Bo] x x [0, B.-1] C R", S n, via projection 9 X + S.
We assign each region S an importance according to target probability measure x. An ergodic cost (z) intuitively measures how well trajectory x E covers S in proportion to w: it penalizes
regions that are under- or over-visited relative their importance. Driving Ex) - 0 therefore
ensures the robot spends time in each region proportional to its assigned importance (see Definition in Appendix D formal introduction on ergodic cost). Our objective compute feasible trajectories that minimize this ergodic cost while satisfying the constraints of the system. Over fixed horizon T, we solve
minimize (()))((X))
(x(())
(2)
(()) vie[u,1]
subject (x) 0, h2(x) V1 E
where initial conditions z00) and trajectory constraints can be accounted for in the equality and inequality constraints h1, h2 e.g., f(z u) and U.
In what follows, we develop strin-variational and evolutionary-strater methods to efficiently generate diverse high-quality ergodic trajectories in parallel.
3 Stein variational ergodic search
SVES reformulates ergodic search problem where robot must explore the domain S according to a
target distribution, see Appendix D) as an inference problem. A binary indicator O =(X) - {0, 1}, flags trajectories that satisfy the optimality criterion. The likelihood of = given a trajectory is defined by p(O = x) exp( p-p(--5 & (x)), where & (ar)) is the ergodic cost and μ > 0 scales its influence. By combining this likelihood with a prior p(x) over the space of trajectories via Bayes' rule, one can obtain the posterior p(x O). We use Stein Variational Gradient Descent (SVGD) [10] (see Appendix B), iterative algorithm approximate this posterior p(2: O) p(æ) exp(-1& with a set of particles so that their empirical distribution approximates the target posterior. At each iteration, particles are nudged toward to regions of high posterior density and while being repelled
from one another to promote coverage, thus forming a sample- approximation of the true posterior
Integrating dynamics and constraints: can be folded into composite cost
The dynamics and constraints hy (xx = 0, h2(x) S 0, etc.,
(x) = Ex()) + p(x) + ei (x)2) + m2x
(3)
Throughout this paper we use the notation C (1,
= {s: + V
((t exists and continuous on for all < و 2). and (L, V) == {/ is continuous).
"Since time is limited resource, it is crucial to penalize excessive revisitations in order to maximize the useful information gathered within finite time budget.

---

with p additional penalties and c; >> 0. The likelihood is then given by p(z) = exp(-H Throughout this paper, we set μ 1.
3.1 SVES state trajectories
We discretize a continuous trajectory æ(t)) into x = [ro....,27 -T- p(x) = (á, I² where & interpolates boundary states. SVGD maintains N "particles", in our case N trajectories {x and updates them via
1]. and chose a Gaussian prior
1
Σ I(xx N
+ Vxk((,,,
where is an arbitrary SVGD iteration. The first SVGD term pulls particles toward low-cost ergodic trajectories (or modes with high density) and is often referred as the attractive force; the second, the gradient of the kernel disperses them to preserve coverage and is referred to as the repulsive force. Therefore, the choice of an effective kernel is essential for good mode coverage. key insight to note is that SVGD allows the N particles to be updated in parallel.
3.2 SVES over control sequences
Similar before, one can optimize over discrete control sequences u luo...., ur-1]. with the discretized dynamics 2(+1 F( u4) zp given, and prior p(u) using SVGD and each control- particle u is updated by
N Σ 1=1
(,())
= N
[u (V logp(u,) - nV un ro)) + V Vuk(u, ))
(5)
In MPC-style loop, usually only the first control action is applied, the state is estimated, and the SVGD optimization is repeated for the shifted horizon.
See Appendix E for more detailed introduction to SVES.
4 Signature Transform and Signature Kernel
The signature (transform) and its corresponding kernel [14] have proven to be highly effective in diverse applications ganngig from robotics to deep learning-wha data is naturally represented as a continuous path [14-19]. Informally, the signature transform of a trajectory can be viewed as analogous the Fourier series, where the trajectory represented We now introduce the signature (transform) of a path/trajectory and its corresponding kernel:
countably infinite sequence.
Definition 1. (Informal) Signature (transform) of path [20]. Consider a continuous path I Rd on an interval I R. For any subinterval [s. C I, the signature of over [s, is defined as the sequence
p(z),,1- = (1, Jesuees dr(ut),...
I dr(u,) @ ... dx((),,.... ...). (6)
Jecurc--cuE<t
where 1 denotes the classical tensor product. This collection of iterated integrals captures the essential features of the path and is invariant under reparameterization (i.e.,p(xx) = sp(x for some reparameterization 0).
Essentially one can view the path signature a canonical linear embedding of any multivariate path into an (infinite) series of iterated integrals. It is injective all non-tree-like paths, meaning two such paths share the same signature (in practice one ensures con-tree-likenee by appending a monotonie "time" coordinate). It also satisfies rime-reversal duality: ((()),,b b & so( jabb 1, where the unit element and 'z (t) = æ(a + b b- t).
One can approximate signature kernel reasonably well with the fruncated signature transform of level N and [21, 22] provide efficient means to compute the truncated signature using Dynamic Programming and approximate it using Random Fourier Features.

---

Definition 2. (Informal) Signature Kernel Let I = ju, u'] and J = [e,va] be two intervals, and consider paths x E (T, R24) and y & CJ The signature kernel (x, T x J + TR. is defined by k (x. = (f(x)),, ((V)t), where <(x), and p(y)e denote the signatures of x and over the intervals [u, s] and [v, t), respectively.
Utilizing the kernel trick introduced in [20], one can efficiently compute the full (untruncated) signature kernel. The kernel trick uses the following result:
Theorem 1. [20] (Informal) Ler I (uu 2'] and == ju, r'1 be intervals, and let x E (I, and y C³ (J, BR.. Then the signature kernel kz,y satisfies the following partial differential equation:
(r.y) = =b((),, ((1)) (r,
Üs Öt
subject to the boundary conditions kill (z. =1 E J, (g,) =1 Vs E.
The resulting PDE formulation of the signature kernel enables its efficient computation using any standard numerical scheme, such as finite difference, finite element, or other suitable methods for hyperbolic equations. See Appendix G.5 for formal introduction to the signature kernel and the reasons why it makes good feature map for trajectories.
5 Promoting diversity in SVES using signature kernels
The kernels used in the previous work on SVES [2] do not take full advantage of the Markovian property of the trajectories. Hence, we propose to directly apply k in equations 4 and 5. Computing the full (N N) Gram matrix for N trajectories of length in IRd via dynamic programming incurs time complexity of O T d"). where e depends on the level of truncation of the signature transform. By exploiting the kernel trick as stated in Theorem 1, the cost of evaluating the kernel between any two paths can computed in low as ( d). Although forming the complete Gram matrix still requires 22 such evaluations and hence retains an (NNT22 overall cost, the underlying PDE formulation admits efficient parallelization. In particular, when implemented with explicit miite-difference scheme on a suitably provisioned GPU, the per-pair computation can driven down to linear dependence on the trajectory length (in practice), yielding an effective wallclock complexity of (T d) under the assumption that the GPU can accommodate the required number of threads [20].
6 Promoting Diversity in SVES using SV-CMA-ES
CMA-ES updates have been demonstrated to yield more efficient search steps than standard gradient descent across a variety of benchmark problems [23-25]. Morcover, [26] empirically established that SV-CMA-ES, a variant of SVGD that uses CMA updates, can surpass SVGD in update efficiency, leading to faster and more accurate convergence the ground truth under certain conditions. In the spirit of these findings, we integrate SV-CMA-ES into the SVES framework in lieu of SVGD. A background in SV-CMA-ES gradient of the log density V log p(z) (a process that typically involves multiple sequential back- and-forth evaluations of the log-density, in practice) which can be very expensive [10]. However, SV-CMA-ES updates involve the evaluation of fitness function via many independent samples, which turn is used to compute Aj ~ a, _k=1 W',sk Mj,k- gradient-free surrogate of V llogph p(a)) (see C Equation 11). Consequently, on massively parallel hardware (e.g.. GPUs), SV-CMA-ES may reduce wall-clock time compared gradient-bart based SVGD, cases where computing the gradient of the log density is very expensive or not available. In the case of Ergodic search, the fitness function f()) in the update (Equation 12) instantiated the ergodic cost J, (-).
be found in Appendix C. SVGD's update requires computing the
scheme further enhances trajectory diversity: it simultaneously samples across multiple subpopulations (exploration) and then refines its search by exploiting a select subset of high-fitness (elite) trajectories.
Moreover, SV-CMA-ES' gradient-fre
Evolution Strategies for Kernel Gradient Estimation: When the analytical gradient of the kernel is not readily available or is computationally intractable, Simultaneous Perturbation Stochastic Approximation (SPSA) may be employed. SPSA is gradient-free technique for estimating gradients
5

---

of black-box functions via simultaneous random perturbations [13]. SPSA, similar to SV-CMA-ES performs multiple, independent kernel evaluations in parallel, thus can also exploit GPU parallelism deliver lower overall wall-clock execution times. We found that the noise from SPSA, in some cases aids in breaking out of "bad" local minima, but in most cases provide no significant performance benefits. By coupling SV-CMA-ES with SPSA, we obtain fully gradient-free algorithm for generating ergodic trajectories. which is the method adopted our experimental evaluation (and hence we will refer to SV-CMA-ES coupled with SPSA just SV-CMA-ES from this point on).
7 Experimental results
In this section, we present results to empirically demonstrate the effectiveness and applicability of the methods discussed above in variety of simulated and real experiments (and their exact setup can be found in Appendix K). Specifically, we are interested in addressing the following:
How diverse are the trajectories produced using the signature kernel?
2. How ergodic are the trajectories produced with the signature kernel?
3. What the convergence rates of SV-CMA-ES and SVGD?
4. How ergodic and diverse the trajectories produced using SV-CMA-ES compared to SVGD?
To address 1) and 2) we will benchmark the signature kernel with other well-known kernels, specifi- cally, the RBF kernel, kerneiized DTW, Global Alignement kernel and the Markov RBF kemel. A brief description of these kernels can found in Appendix G. At each iteration of the SVGD or SV-CMA-EB EES new bandwidth for the kemells was chosen using the median heuristic. We employed the median heuristic for bandwidth selection because is straightforward, computationally light, and requires no manual tuning, making our method more accessible to users (refer to [27] for details on the guarantees provided by the median heuristic).
RBF Kernel
(b) Kernelized DTW
(c) Signature kernel
Figure 2: Top down projection of trajectories generated in experiment comparing kernels in section 1.2.1. Despite using a Gaussian prior that just interpolates start and goal, only the signature kernel's strong repulsive term steers trajectories around obstacles.
The diversity of N trajectories is measured by det(Ky) )). where K2 E NxN is the Gram matrix with entries Ky)ı; kracchel (X, Xy), where krrschet is kernel build using the Fréchet distance. See Appendix H for details.
7.1
Multiscale constrained exploration
7.1.1
Signature kernel performance
To evaluate the performance of the signature kernel within the SVES framework (using SVGD), we instantiate ten distinct priors (+(() for k 1, ... 10 detailed in Section 3.1 & E.2, each with variance x² 0.01. For each prior, we generate N trajectory samples and compare the kernels according to the following criteria in SVES:

---

1) Average trajectory cost, 1 N E Tx (x()..
2) Best trajectory cost, minigigN Jn (x').
3) Mean ergodic cost, N Ei= (xx).
4) Trajectory diversity. - log (de(((())))
All Stein variational gradient updates employ step size of 0.01, and iterations terminated when (ii - ((i-1) < 1.25 x 10-3 or maximum of 3000 iterations is reached. The experimental results are summarized in Figure 3 and elaborate statistical analysis can be found in Appendix (Table 2).
3.39c+00
exp(Best trajectory cost): lower better
2.830 00
explAverage trajectory cost): lower is better
3.0
exp(Mean ergodic cost): lower is better
Trajectory diversity: higher is better
Average Metric Value over 10 priors
2.5
2:33e-00
177e-000
10te400
16ler00
1.0e-c0
1.00-000
173e100
00
1.5
12le+00
3.30e-40 00
200 -00
1.0
Global Alignment Kemel
Signature Kernel
Kernelized DTW
Markov RBF
RBF
Figure 3: Aggregated performance comparison across five kernel categories. For each kernel, the four side-by-side bars represent the average of 10 experimental runs for the metrics discussed in 7.1.1.
7.1.2 SV-CMA-ES vs SVGD
To assess the comparative performance of SV-CMA-ES and standard SVGD on the ergodic trajectory optimization, we initialize both methods with N trajectories drawn from the same Gaussian prior (variance x² 0.01). All other hyperparameters are held constant. Here we use the RBF kernel. At each optimization iteration, we record:
1) Average trajectory cost, 1/4 ΣN (x')..
2) Best trajectory cost minigigN Iz (x').
3) Trajectory diversity, - logg(1 - dkt(
Figure illustrates the per-iteration evolution of these metrics. SV-CMA-FS maintains similar costs compared to SVGD, while promoting trajectory diversity.
7.2
3D Coverage using the Crazyflic drone
7.2.1
Signature kernel performance
We follow the exact method used in section 7.1.1, but in example here N 10, 0.02, and iterations are terminated when X(i) x((= < 1.25 x 10-3 or a maximum of 1500 iterations is reached.
The experimental results are summarized in Figure and an elaborate statistical analysis can be found in Appendix J (Table 3).
7

---

(Average Trajectory Cood)
SYCHA-ES
10
SV ES 0.2
log (Best Trajectory Cost)
SV CMA ES 0.5
Trajectory Diversity
SYGD
SYGD
800
1600 Num. iter.
2400
800
1600 Num, Iter.
2400
1600
2400
Num. Iter.
Figure 4: Convergence results for different step sizes (defined for SVES and ag for SV-CMA-ES in Appendices B & C respectively) for the multiscale constrained exploration experiment.
exp(Best trajectory cost): lower is better
100e+01
explAverage trajectory cost): lower is better
exp(Mean ergodic cost): lower is better
89t++00
Average Metric Value over 10 priors
exp(Trajectory diversity): higher better
8
7,30e+00
7.03e+000 00
7
3
5.0ee+00 3.00r000
2 00
27
3000
2,33e+00
2.04 00
1330 00
1.77x000
1.75e100
2
RBF
Global Alignment Kernel
Kernelized DTW
Markov RBF
Signature Kemnel
Kerelss
Figure 5: Aggregated performance comparison across five kernel categories. For each kernel, the four side-by-side bars represent the average of 10 experimental runs for the metrics discussed in 7.2.1.
7.2.2
SV-CMA-ES vs SVGD
We once again follow the method in section 7.1.2 with N trajectorics. Figure illustrates the per-iteration evolution of these metrics.
7.3
Stein variational model-predictive control
7.3.1 Signature Kernel Performance
We instantiate ten distinct priors pe(.) for k= 1,....... 1,...., 10 similar to 7.1.1. For each prior, we record
the following each model predictive control (MPC) iteration 1,..., 200.
1) Average trajectory cost,
2) Best trajectory cost,
fi:= 인 ΣN (u').
Js := min J4(u')
3) Ergodic cost of the best trajectory, g : 3d (u,):
4) Trajectory diversity.
D log K')
arg min (ux)

---

(Average Trajectory Coog
1.5
SVCMMEES 0.1
5V-CMAES 0.2
(Best Trajectory Cost)
1.5
SV ES 0.5
SVCCAAE5
1.0
20:
0.5
SVGD 0.1
SYCD 0.2
0.5
Trajectory
2400
1600 Num. Iter.
2400
1600
2400
Num. Iter.
Num. Iter.
Figure 6: Convergence results for different step sizes (defined for SVES and ae for SV-CMA-ES in Appendices B & C respectively) for the 3D coverage using the Crazyflic drone experiment.
Table 1: Aggregated metrics over all time steps for single prior
RBF
Signature
Markov
Kernelized
kernel
RBF
DTW
Total average trajectory cost: Σ F
563.14
581.27
764.08
554.92
Total best trajectory cost: 2,20 Jo+
377.83
351.83
468.82
375.13
Total ergodic cost of best trajectory:
ΣØ=1
200
Ehi
262.09
239.73
319.88
262.09
Total trajectory diversity: Σ)=1 D
200
48.51
548.22
3395,45
57.46
The experimental results for such prior summarized Table 10 priors are summarized in Appendix Table
The experimental results for all
Markov RBF failed to converge to meaningful solution in most of the MPC iterations resultin in essentially random trajectories and thus artificially high diversity values.
7.4 Ablations: kernel bandwidth sensitivity
We provide small ablation study on the effect of the choice of bandwidth on kernels in the 3D Coverage using the Crazyflie drone experiment from Section 7.2. Figure 7 compares different computationally light heuristics [28] (mean heuristic, Sliverman's 's rule of thumb and Scott's 's rule) for selecting the bandwidth and Figure shows the effect of manually tuning the kernel.
Silverman's rule--o fule-of-thumb
Mean heuristic
Scott's rule
37.5
expiirest
Avarage Matric Voluue over 10 pilnts
expi4ees costs lower
eitectery diveriity figner better
32.5
10.0
Averape Matric Value
GI000 A Algennenc HOrKYy
RDP
Glool
Algremenc Manoy
Sqnature
HSP
Rigemank Markos
Kernes
Gernels
Kernels
Figure 7: The 3D coverage using the Crazyflie drone experiment with different kernel bandwidth selection heuristics. The plots report the mean over 10 random priors, with error bars indicating +1 standard deviation.

---

135
130
1.29
3:55
2.0
3.05
RCF
RB
Parton RET
Vetoo
603
Excntie
625
0425
e.t
Bandwitth
Bendwith
Bandwitdh best trajectory cost
(b) Bandwidth vs average trajectory cost
agrauu and!
Same
Geea xigmert tamel
Gata régrnent
eppaeenn egont core
Gervibec DTM
DTW
2.00
Pejecteyy
200
650
adl
ezs
6ats
Got
625
Bardwiith
Bandwltth
(c) Bandwidth vs mcan ergodic cost
(d) Bandwidth vs trajectory diversity
Figure 8: Impact of bandwidth selection on kernel performance 3D coverage using the Crazyflie drone experiment. Each sub-figure presents the mean over random priors, with error bars indicating +1 standard deviation.
8 Conclusion, Limitations and best practices
In summary, integrating the signature kernel into ergodic search framework consistently lowers ergodic cost and increases trajectory diversity, compared to standard choices by promoting more effective mode coverage. Moreover, swapping in SV-CMA-ES further enhances exploration diversity, while completely being a gradient-free approach, underscoring the practical benefits of our approach. See Appendix J for the visual representation of the trajectories.
However our approach has several limitations. First, the runtime of the methods discussed this paper depends heavily on the available CPU/GPU parallelism and the dimensionality of the problem: as the number of particles and dimensions increases, both communication and computation overhead grow, which can degrade performance very high dimensions. Second, the SV-CMA-ES instantiation usually requires performing a singular value decomposition (SVD) Cholesky factorization every iteration. Consequently. the scalability (and hence the real time applicability) of this method depends heavily on the problem size, use case, cost of computing the gradients, available hardware etc., and practitioners should be aware of these trade-offs. Future work could explore incorporating recent advances (e.g., diagonal covariance matrices [29]) in vanilla CMA improve efficiency. Third, while the signature kernel provides very strong repulsive force that promotes diverse exploration, it can also inadvertently push trajectories into obstacles, outside bounded domains, etc. We noticed this specially the example in section 7.3, which explains the higher total average trajectory cost. To mitigate these effects, practitioners should consider using more conservative cost designs when employing the signature kernel, opting for alternative kernels (e.g., RBF) that exert milder repulsion or running SVGD for more iterations (see Appendix for the exact convergence guarantees; we also recommend the reader refer to [2, 30]). Fourth, SV-CMA-ES does not provide rich theoretical guarantees provided by SVGD and hence SVES. Finally, we did not explore the potential of the Fréchet Distance kernel for producing diverse trajectories due to its computational cost.
hardware with limited cores/threads when scaling to
10

---

References
[1] L. M. Miller, Y. Silverman, M. A. Maclver, and T. D. Murphey, "Ergodic exploration of distributed information,"" IEEE Transactions Roboties, vol. 32, pp. 36-52, 2016.
[2] D. Lee, C. Lerch, F. Ramos, and I. Abraham, "Stein variational ergodic search," in Robotics: Science and Systems XX, Delft, The Netherlands, July 15-19, 2024, D. Kulic, G. Venture, K. E. Bekris, and E. Coronado, Eds., 2024. [Online]. Available: https://dii.or//10. wttp:/orrr/....51///......
[3] D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell, "Curiosity-driven exploration by self- supervised prediction," in Proceedings of the 34th International Conference on Machine Learn- ing- Volume 70, ICML'17. JMLR.org, 2017, 2778-2787.
[4] Y. Burda, H. Edwards, D. Pathak, J. Storkey, T. Darrell, and A. A. Efros, "Large-scale study of curiosity-driven learning." in 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. [Online]. Available:
[5] P. Mazzaglia, O. Catal. T. Verbelen, and B. Dhoedt, "Curiosity-driven exploration via latent bayesian surprise," 2022. [Online]. Available: http::/arrai..og///b21110044455
[6] Y. Silverman, M. Miller, M. A. MacIver, and T. D. Murphey, "Optimal planning for infor- mation acquisition." in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 1974-5980.
[7] W. Chen, R. Khardon, and L. Liu, "Ak: Attentive kernel for information gathering," 06 2022.
[8] G. Mathew and I. Mezić, "Metrics for ergodicity and design of ergodic dynamics for multi-agent systems," Physica Nonlinear Phenomena, vol. 240, no. pp. 432-442, 2011. [Online]. Available:
[9] L. M. Miller and T. D. Murphey, "Trajectory optimization for continuous ergodic exploration." in 2013 American Control Conference, 2013, pp. 4196-4201.
[10] Q. Liu and D. Wang, "Stein variational gradient descent: A general purpose bayesian inference algorithm," in Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg. Guyon, and R. Garnett, Eds., vol. 29. Curran Associates, Inc., 2016. [Online]. Available: beba8flbee 213-211376339005888888 Papp
files/paper/2016/itte/
[11) I.. Barcelos, T. Lai, R. Oliveira, P. Borges, and F. Ramos, "Path signatures for diversity probabilistic trajectory optimisation," The International Journal of Robotics Research, vol. 43, no. 11, pp. 1693-1710, 2024. [Online]. Available:
[12] C. Braun, R. Lange, and M. Toussaint, "Stein variational evolution strategies," " 10 2024.
[13] J. Spall, "Multivariate stochastic approximation using simultaneous perturbation gradient approximation." IEEE Transactions on Automatic Control, vol. 37, no. 3, pp. 332 332-341, 1992.
[14] D. Lee and H. Oberhauser, "The signature kernel," 2023. [Online]. Available: https://axivvorg/bbb//0050000000
[15] I. Chevyrev and A. Kormilitzin, "A primer the signature method in machine learning," 03 2016.
[16] P. Bomnier, P. Kidger, I. Perez Arribas, C. Salvi, and T. Lyons, "Deep signature transforms," 05 2019.
[17] W. Yang, T. Lyons, Ni, C. Schmid, L. Jin, Developing the Path Signature Methodology and Its Application to Landmurk- Based Human Action Recognition. Cham: Springer International Publishing, 2022, pp- 431-464, [Online]. Available: http:://o.arr//oo.0777934--00013

---

[18] M. Ohnishi, I. Akinola, J. Xu, A. Mandlekar, and F. Ramos, "Signatures meet dynamic programming: Generalizing Bellman equations for trajectory following," in Proceedings of the 6th Annual Learning for Dynamics amp; Control Conference, ser. Proceedings of Machine Learning Research, A. Abate, M. Cannon, K. Margellos, and A. Papachristoodoulou, Eds., vol. 242. PMLR, 15-17 Jul 2024, pp. 466-479. [Online]. Available:
[19] Z. Yin, T. Lai, L. Barcelos, J. Jacob, Li, and F. Ramos, "Diverse motion planning with stein diffusion trajectory inference,"" in 2025 IEEE International Conference on Robotics and Automation (ICRA), 2025, pp. 15 610-15616.
[20] C. Salvi, T. Cass, J. Foster, T. Lyons, and W. Yang. "The signature kernel is the solution of a goursat pde," SIAM Journal Mathematics of Data Science, vol. 3, no. 3, pp. 873-899, 2021. [Online]. Available: https://doi.org/ 10.1 0.11377/000136679
[21] J. Kiraly and H. Oberhauser, "Kernels for sequentially ordered data," Journal of Machine Learning Research, vol. 20, no. 31, pp. 1-45, 2019. [Online]. Available:
htpp://mloorrg/pppprs////1111111
[22] C. Tóth, H. Oberhauser, and Z. Szabó, "Random fourier signature features," " SIAM Journal on Mathematics of Data Science, vol. 7, no. 1, pp. 329-354, 2025. [Online]. Available:
[23] J. Martens, "New insights and perspectives on the natural gradient method," Journal of Machine Learning Research, vol. 21, no. 146, pp. 1-76, 2020. [Online]. Available: http://mlt..org/aaerr///// 11-8..78ml
[24] Y. Akimoto, Y. Nagata, I. Ono, and S. Kobayashi, "Theoretical foundation for cma-es from information geometry perspective," Algorithmica, vol. 64, pp. 698-716, 2012. [Online]. Available:
[25] T. Glasmachers,
T. Schaul, S. Yi, D. Wierstra, and J. Schmidhuber, "Exponential natural evolution strategies," in Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, ser. GECCO 10. New York, NY, USA: Association for Computing Machinery, 2010, p. 393-400. [Online]. Available: htts://doi.org/10.. 144/1413044331833333
[26] C. V. Braun, R. T. Lange, and M. Toussaint, "Stein variational evolution strategies," 2024. [Online]. Available: https://ariv..or/////441....
[27] D. Garreau, W. Jitkrittum, and M. Kanagawa, "Large sample analysis of the median heuristic," 2018. [Online]. Available: https://arxiv.or//ab///110
[28] S. Sheather, "Density estimation." Statistical Science, vol. 19, 11 2004.
[29] R. Ros and N. Hansen, "A simple modification in ema-es achieving linear time and space complexity." in Parallel Problem Solving from Nature PPSN X, G. Rudolph, T. Jansen, N. Beume, S. Lucas, and C. Poloni, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg. 2008, pp. 296-305.
[30] A. Korba, A. Salim, M. Arbel, G. Luise, and A. Gretton, "A non-asymptotic analysis for stein variational gradient descent," " 2021. [Online]. Available: https://axxi..og//bss2200...
[31] N. Ratliff, M. Zucker, J. A. Bagnell, and S. Srinivasa, "Chomp: Gradient optimization tech- niques for efficient motion planning," in 2009 IEEE International Conference on Robotics and Automation, 2009, pp. 489 494.
[32] M. Zucker, N. Ratliff, A. D. Dragan, M. Pivtoraiko, M. Klingensmith, C. M. Dellin, A. Bagnell, and S. S. Srinivasa, "Chomp: Covariant hamiltonian optimization for motion planning." The International Journal of Robotics Research, vol. 32, no. 9-10, pp- 1164-1193, 2013. [Online). Available:
[33] A. Byravan, B. Boots, S. S. Srinivasa, and D. Fox, "Space-time functional gradient optimization for motion planning," " in 2014 IEEE International Conference on Robotics and Automation (ICRA), 2014, pp. 6499-6506.
12

---

[48] D. L. Applegate, R. E. Bixby, V. Chvatál, and W. J. Cook, The Traveling Salesman Problem: A Computational Study. Princeton University Press, 2006. [Online]. Available: http://www..tooooogsttbbee...t.t [34] Z. Marinho, A. Dragan, A. Byravan, B. Boots, S. Srinivasa, and G. Gordon, "Functional gradient motion planning in reproducing keree hilbert spaces," 2016. [Online]. Available: https://arxiv.orr/bbb/// 0et////661000.44
[35] M. Kalakrishnan, S. Chitta, E. Theodorou, P. Pastor, and S. Schaal, "Stomp: Stochastic trajectory optimization for motion planning," in 2011 IEEE International Conference on Robotics and Automation, 2011, I, pp. 4569-4574.
[36] N. Hansen, S. D. Müller, and Koumoutsakos, "Reducing the time complexity of the derandom- ized evolution strategy with covariance matrix adaptation (cma-es)," Evolutionary Computation, vol. 11, no. I, pp. 1-18, 2003.
[37] P. Hämäläinen, A. Babadi, X. Ma, and J. Lehtinen, "Ppo-ema: Proximal policy optimization with covariance matrix adaptation." " in 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing (MLSP), 2020, pp. 1-6.
[38] B. Tjanaka, M. C. Fontaine, D. H. Lee, A. Kalkar, and S. Nikolaidis, "Training diverse high-dimensional controllers by scaling covariance matrix adaptation map-annealing," " 2023. [Online]. Available: https://ariv.orr/aa///0...222
[39] N. Hansen, "The cma evolution strategy: A tutorial," 2023. [Online]. Available:
https//aaxii.org/bbb//66..4 604.00007
[40] Schulman, J. Ho, A. Lee, I. Awwal, H. Bradlow, and P. Abbeel, "Finding locally optimal, collision-fie free trajectories with sequential convex optimization," 06 2013.
[41] M. Mukadam, J. Dong, X. Yan. F. Dellaert, and B. Boots, "Continuous-tmme gaussian process motion planning via probabilistic inference," " The International Journal of Robotics Research, vol. 37, no. 11, pp. 1319-1340, 2018. [Online]. Available: http:///ci.rr/1111177/777844
[42] J. Dong, M. Mukadam, F. Dellaert, and B. Boots, "Motion planning as probabilistic inference using gaussian processes and factor graphs," " 06 2016.
[43] M. Mukadam, J. Dong, F. Dellaert, and B. Boots, "Steap: simultaneous trajectory estimation and planning," " Autonomous Robots, vol. 43, no. 2, p. 415-434, Jul. 2018. [Online]. Available: http://dx.doi.org)
[44] A. Lambert and B. Boots, "Entropy regularized motion planning via stein variational inference," 2021. [Online]. Available: https:/aariv.oor/bb2221.....
[45] H. Yu and Y. Chen, "A gaussian variational inference approach to motion planning," IEEE Robotics and Automation Letters, vol. no. 5, p. 2518-2525, May 2023. [Online]. Available:
http://dx.doi.or//10.. wtt.
[46] A. Lambert and B. Boots, "Entropy regularized motion planning via stein variational inference," 2021. [Online]. Available: http://arwivvarr////2/177.... 10705514
[47] H. Choset, "Coverage of known spaces: The boustrophedon cellular decomposition." Auton. Robots, vol. 9, pp. 247-253, 12 2000.
[49] G. Laporte, H. Mercure, and Y. Nobert, "Generalized travelling salesman problem through n sets of nodes: the asymmetrical case," Discrete Applied Mathematics, vol. 18, no. 2. pp. 185-197, 1987. [Online]. Available: 0166218X87900205
[50] G. Chirikjian, Stochastic Models, Information Theory, and Lie Groups, Volume 2: Analytic Methods and Modern Applications, ser. Applied and Numerical Harmonic Analysis. Birkhäuser Boston, 201 I. [Online]. Available: https//ooooooooooooooooooooo/oo/
13

---

[51] D. Dong, H. Berger, and I. Abraham, "Time optimal ergodic search," 2023. [Online]. Available: https:/arxiv.oor/bbb/233.....
[52] S. E. Scott, T. C. Redd, L. Kuznetsov, I. Mezić, and C. K. Jones, "Capturing deviation from ergodicity at different scales," " Physica D: Nonlinear Phenomena, vol. 238, no. 16, pp. 1679, 2009. [Online]. Available: http://wwwwceeeendddeooo//////ee $0167278909001523
[53] L. Dressel and M. Kochenderfer. "On the optimality of ergodic trajectories for information gathering tasks," in 2018 Annual American Control Conference (ACC), 2018, pp. 1855-1861.
[54] T. Howell, N. Gileadi, S. Tunyasuvunakool, K. Zakka, T. Erez, and Y. Tassa, "Predictive sampling: Real-time behaviour synthesis with mujoco," 12 2022.
[55] G. Williams, P. Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou, "Aggressive driving with model predictive path integral control," in 2016 IEEE International Conference Robotics and Automation (ICRA), 2016, pp. 1433-1440.
[56] G. Williams, A. Aldrich, and E. A. Theodorou, "Model predictive path integral control: From theory to parallel computation," Journal of Guidance, Control, and Dynamics, vol. 40, no. 2, pp. 344-352 2017. [Online]. Available:
[57] M. Sun, F. Baldini, P. Trautman, and T. Murphey, "Move beyond trajectories: Distribution space coupling for crowd navigation," " 07 2021.
[58] A. Lambert, F. Ramos, B. Boots, D. Fox, and A. Fishman, "Stein variational model predictive control," in Proceedings of the 2020 Conference on Robot Learning, ser. Proceedings of Machine Learning Research, J. Kober, F. Ramos, and C. Tomlin, Eds., vol. 155. PMLR, 16-18 Nov 2021, pp. 1278-1297. [Online]. Available: https:/proceedings.ll..p. p5sma/////////////////////// 155/lamber22 2 a.hhml
[59] A. Le, G. Chalvatzaki, A. Biess, and J. Peters, "Accelerating motion planning via optimal transport," 10 2023.
[60] C. Hughes, H. Warren, D. Lee, F. Ramos, and I. Abraham, "Ergodic trajectory optimization on generalized domains using maximum mean discrepancy." CoRR, vol. abs/2410.10099,, 2024. [Online). Available: http://bbb..r//1..4555/////a////
0.10999
[61] Power and Berenson, "Constrained stein variational trajectory optimization," " Trans. Rob., vol. 40, p. 3602-3619, Jan. 2024. [Online]. Available: https://doi.org/10..1 119.////.0.......10000000..0..00 109/TRO.2044. 3428428
[62] B. Ghojogh, A. Ghodsi, F. Karray, and M. Crowley, "Reproducing kernel hilbert space, mercer's theorem, eigenfunctions, nyström method, and use of kernels in machine learning: Tutorial and survey," 2021. [Online]. Available: http://arxiv.or//abb/22 http//////10.044422.. 06..0444
[63] M. Cuturi, J.-P. Vert, O. Birkenes, and T. Matsui, "A kernel for time series based on global alignments," " 05 2007, pp. II-413.
[64] P.-F. Marteau and S. Gibet, "On recursive edit distance kernels with application to time series classification," " IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. p. 1 1121-1133, Jun. 2015. [Online]. Available: http://dx.doi.oor/1100 1ttp:///....1111....00.002223320
[65] D. Haussler, "Convolution kernels on discrete structures uusc-crr--9--0; 06 2001.
[66] The Signature of a Path. Berlin, Heidelberg: Springer Berlin Heidelberg, 2007, pp. 25-40.
[Online]. Available: https://doi.or// 10.
[67] T. Lyons, "Rough paths, signatures and the modelling of functions on streams," 05 2014.
[68] M. Fréchet, Sur quelques points de calcul fonctionnel.
na, 1906. [Online]. Available:
[69] N. Megiddo, "Applying parallel computation algorithms in the design of serial algorithms." J. ACM, vol. 30. no. 4, p. 852-865. Oct. 1983. [Online]. Available: http://...oorg...111111151.....
[70] R. Cole, "Parallel merge sort," SIAM Journal Computing, vol. 17, no. 4, pp. 770-785, 1988.
14

---

Appendices
Appendix Contents
Additional related work
15
Stein variational gradient descent (SVGD)
16
C Stein variational covariance matrix adaptation evolution strategy
17
D Ergodic search and ergodic trajectory optimization
19
Stein variational ergodic search
F Convergence of SVGD in the Ergodic Search Setting
23
G Kernels
24
Measuring trajectory diversity
27
I Fréchet distance
28
J Additional results
28
K Implementational and experimental deatils
33
A Additional related work
Trajectory optimization encompasses a wide range of techniques that initiate with a preliminary, suboptimal path and iteratively refine to converge on (potentially local) optimum by minimizing associated cost function. Foundational contributions in this domain include Covariant Hamiltonian Optimization for Motion Planning (CHOMP) [31] and its related approaches [32-34]. which exploit the covariance of trajectories alongside Hamiltonian Monte Carlo to perform annealed functional gradient descent. A significant drawback of these methods is their reliance on fully differentiable cost functions. Stochastic Trajectory Optimization for Motion Planning (STOMP) [35] overcomes this by estimating gradients from stochastic samples of noisy trajectories, thereby accommodating cost functions that lack differentiabilityy
Trajectory optimization
Another influential research direction involves quality diversity algorithms, notably the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) [36-38], which adopts a derivative-free strategy using multivariate normal distributions to generate and update candidate solutions. Although CMA- ES is robust and exhibits ergodic behavior capable of managing multimodal problems, it typically requires multiple initializations and higher number of evaluations compared to gradient- based methods [39]. Other techniques, such as TrajOpt [40], solve sequential quadratic programs with continuous- time collision checking; while these methods are efficient, they often yield only locally optimal solutions and necessitate fine parametrization of waypoints. To mitigate this, [34] limit the optimization to a Reproducing Kernel Hilbert Space (RKHS) with a although this may overlook the cost contributions between sparsely distributed waypoints. Methods proposed in GPMP [41-43] model trajectories as Gaussian Processes (GP) to obtain maximum a posteriori (MAP) solution.
kernel,
More closely aligned with our approach, variational inference methods introduced in [44] and [45] infer the posterior distribution over trajectories using Stein variational gradient descent (SVGD) or
15

---

natural gradient updates. In contrast, the RKHS induced in SVMP fails to capture the sequential dynamics of trajectories by treating them as static, high-dimensionas vectors-resute restli--esuuling in reduced repulsive forces and less coordination across functional dimensions. Kernel Signature Variational Gradient Descent (SigSVGD) [11], overcomes these shortcomings by integrating path signatures to explicitly encode the sequential structure of trajectories. This approach not only enhances diversity but also increases the probability of discovering superior local optima. Furthermore, leveraging insights from both motion planning and -antro--a--nfferenne allows SigSVGD to extend naturally to Model Predictive Control (MPC) frameworks (e.g.. Stein Variational MPC (SVMPC) [46]), where preserving the sequential structure of control trajectories is crucial.
Robotic exploration Robotic exploration is primarily concerned with steering agents into unknown territories, while the coverage problem entails designing trajectories and strategically placing sensors to ensure that confined area is thoroughly examined. Early solutions often relied on discrete grid-based serateicss-mpllyye antecedent to the lawnmower pattern [47]) and variations of the traveling salesperson problem [48, -9 to list all viable exploration routes. Although grid-based methods offer guarantees of completeness, scaling these techniques to continuous domains is inherently difficult because the number of potential trajectories becomes unbounded. Recent strategies in reinforcement learning and information-driven exploration have sought to surmount these obstacles by using "curiosity" measures [3. 4] derived from information theory [50]. However, many of these approaches tend to be short-sighted, concentrating on immediate rewards and often settling on a single-mode solution, which restricts the system's 's overall flexibility.
methods such as advanced boustrophedon search patterns (the
Subsequent developments have brought forth ergodicity-basee methods for exploration. By reducing the disparity between the expected spatial information distribution and the time-averaged visitation of path [8], ergodic exploration techniques yield sophisticated coverage patterns across continuous domains [1, 51]. In fact, the spectral, multi-scale structure of the ergodic metric [52] suggests the existence of numerous exploitable local solutions-an element that is essential for resilient, real-time exploration [53]. Despite these advances, a persistent challenge remains in devising methods that compute sets of "good," locally optimal ergodic trajectories which can facilitate efficient strategy switching in dynamic scenarios.
Control as inference The control as inference framework has recently emerged as a powerful perspective by recasting trajectory optimization issues within a probabilistic framework. The inherent non-convexity of many robotic problems-due to complex dynamics and contact cnteractions-ereettes significant hurdles for traditional gradient-based optimization techniques. Approaches such as predictive sampling [54] and podel-predictive path integral control (MPPI) [55, 56] have been developed mitigate these difficulties. However, these methods often produce only a single locally optimal solution, thereby overlooking the potential for multiple, equally effective strategies. This limitation is especially problematic in rapidly changing environments for instance, in crowd-based aoggation-wheer relying on one trajectory can be inadequate [57, 58]. Moreover, the utilization of optimal transport and variational techniques has shown promise by reformulating trajectory optimization as an inference problem [58, 59]. These frameworks enable the approximation of complex solution distributions that encompass multiple local optima, thus providing the necessary flexibility for swift adaptation.
B Stein variational gradient descent (SVGD)
Variational inference often requires approximating an intractable posterior distribution p(x) by a tractable distribution qxx) from a parameterized family of distributions D. typically through the minimization of the Kullback-Leibler (KL) divergence. Stein Variational Gradient Descent (SVGD) [10] offers a nor-parametric approach that iteratively transports a set of particles {r,)²_. to match the target distribution.
The core idea of SVGD to gradually push the particles in the direction that most rapidly decreases the KL divergence. Consider an incremental, smooth transform
T(x) = c((x),
16

---

where 6(x) is smooth perturbation function and > 0 is small step size. A key theoretical result shows that the derivative of the KL divergence with respect (evaluated at 0) is
d KL (LT
lI2)
--Ean trace (And(z))
de
c=0
with the Stein operator defined as
Apo(x) V log p(z) 6(x) T + Vz0(z).
This result, based on Stein's 's identity, motivates the introduction of the kernelized Stein discrepancy (KSD) as a measure of the mismatch between (zx) and p(z).
By constraining lie within the unit ball of reproducing kernel Hilbert space (RKHS) associated with a positive definite kernel *(),,), one can derive a closed-form expression for the optimal perturbation direction:
(-) = Bawg Vlogp())) Vak(z,))
8
Here, the first term the expectation directs particles toward regions of high probability under p(æ). while the second term serves as a repulsive force, ensuring that the particles remain diverse and do collapse into single mode.
The practical SVGD update for the particles is then given by
+ e 2² (I;),
(9)
where the empirical approximation of the optimal direction is
1 n2 Σ (((z,,)VV logp(1;) VV [(r,
(x)))
(10)
Notably, when n = the update reduces to the standard gradient ascent used in maximum a posteriori (MAP) estimation, whereas using multiple particles allows SVGD capture the full posterior distribution.
SVGD's formulation functional gradient descent in RKHS not only avoids the need to compute Jacobians or their inverses but also scales well with large datasets by permitting mini-bate batch approximations of V p(x) and allows for parallel computation of each particle. This scalability, along with its ability to maintain particle diversity through the repulsive kernel term, makes SVGD an attractive and efficient method for Bayesian inference- and it forms key component in extensions to more complex tasks.
C Stein variational covariance matrix adaptation evolution strategy
To overcome the reliance on gradient information in standard SVGD while retaining its powerful repulsive interactions, Stein Variational Covariance Matrix Adaptation Evolution Strategy (SV-CMA- ES) embeds the adaptive, intural-gradient-feee CMA-ES into the multi- martii- SVGD framework [26]. Conceptually, each SVGD "particle" is no longer point but the mean of its own Gaussian search distribution. By interpreting the CMA-ES mean update as an estimate of the score function v log p(x), we obtain gradient-free yet highly explorative algorithm that tracks both local fitness landscapes and global diversity.
Let n be the number of particles and m the sub-population size per particle. Particle carries Gaussian
Fi Ei,
Si, ~ N(x,, C,), y1.6
k= 1, ,,
and we evaluate their fitness f(E.,)). By ranking these samples and selecting the top À as the elite sub-population, we assign recombination weights wi, (non-7ero only for those À elites, which may be positive negative depending on their ranked fitness; see [39]).
17

---

In vanilla SVGD each particle moves according to
34 - Jj + 6 T2 Rx
V Vlogp(s) k(x, + Vak(r, x.)]
In SV-CMA-ES replace the intractable score by the CMA-ES mean shift
4, Σ wjsk (E,. 2z) = J
(11)
and approximate the expectation over all Gaussians. The resulting SV-CMA-ES update is
n Σ j=1
O n
(o,, V Vo,, k(z,,
(12)
where 0g subsumes the original SVGD step-size e. This blend preserves SVGD's repulsion through Vk while guiding particles toward high-fitnew figh-fitness regions via CMA-ES's Az.
The self-adaptation o, and C2 follows CMA-ES's mechanisms:
Aef
(13)
I|Po
h4
(14)
(1 ag )2((+1)*
1, hi < (1.4+ a BIIA(O,
ba.
(15)
0, otherwise,
df (ha.s) = - ho,s) ae (2 - ac)
(16)
Wiikr
Wi,k 2 0,
Wi,k
(17)
Wi.k d ||c;;
112
1/2 yi.k
Wi,k < 0.
an a2)) ax) C
A
Poç f- (1 an) Poçi
(18)
Aoff
ag
- -1))
T f- o, exp
(19)
do
and
a ac)
A
Pe,i f- - (1-aa) Pe.s + ho,s
(20)
Aell O₃
C2 (1+ a1
as Σ Wi,k) C
01
Pe,iPes
ax
Σ
Tr V.
(21)
Practical considerations
The driving-foree term in (12) averages over all n particles, which can shrink steps when modes are distant and cause premature step-size reduction. To address this, one may employ hybrid update that uses only the particle's own CMA-ES shift for the first term and retains the full repulsion:
1 W. (Es. x.) rtt)
24 < I + Oke
V 2, k(z,,
(22)
n
where 7()) is an annealing factor (e.g. 7(1) maxxlog T. 1)) that gradually reduces repulsion to fine-tune convergence near local optima and prevents premature convergence.
18

---

Algorithm Stein Variational CMA-ES
Require: kernel ( )) sub-suubpuo sub-population size m; elite size X; hyper- parameters Cz a any Oeey do; n particles; 7 iterations
Initialize: (Iit, Ci) for ,,...,n
2: for 1, T do
outer optimisation loop b one search distribution per particle
3: for = 1,................ do
Sample sub- sub-population
4:
for j = 1...., m do
5:
Eu, Mz
Vi.j - 2 6: end for
7:
Evaluate & accumulate statistics
8:
compute f(S..) and ranks - weights wij
9:
A
Wiik (Ei, xz:)
10:
Ay k().. + VZz 2(3,(,,,3;;)
1=1
Update mean
11:
24 f- I; + Or 9/i
Step-size (CSA)
Aeft &-
Σ
m
W
-1
13:
Po,s t (1
ao Po,i +
a2 aa)
2,2/2 01
dell
14:
ai t exp Og do
IPaall
-1)
Covariance path & matrix compute hg and d(ho.s) (CSA rule)
an - ate)
16:
Po, + (1- - ac)ee,, + havl
D1
Aef
17:
C. (1+1 d(ho,s) -ax
Σ
Wis) C.+ 01 Pe + as Σ Wis Vi.jVie
j=1
18:
end for
19:
end for
All loops other than the outer optimization loop can be parallelized.
D Ergodic search and ergodic trajectory optimization
We denote the state space robot as C and the control space as C W, where and U are some Banach spaces. We also denote the the continuous-time dynamics of the robot as f(z, u): X x + IX. where I denotes the tangent bundle the state-space X. Next, a robot's space trajectory xtf) R20 + X is defined as the solution to the following initial value problem
x(() = x J [(x()) u(E)) de.
(23)
with initial condition (0)) E X, a control trajectory u(t) : R20 - U. addition, we will denote the bounded (and compact) domain (with bounds B,) where the robot explores as S
[0, Bo] x... x [0, B,-1], where v S n. Let g(x) X - S be map that projects state space X' to exploration space S.
19

---

Definition 3. Time-Averaged Trajectory Statistics [2]. Let L be the Lebesgue measure on Rz0 thenfor each T Rzo. the probability measure Vr on S that defines the time-averaged trajectory visitation statistics integrated along time [0, T) is defined by
Mr(A) = T ((go ((((gg))'(())))))) n
(24)
for a A C. C.S is a Borel set.
Definition 4. Ergodicity [2, 9, 8). A trajectory x(t) is said to be ergodic with respect to Borel probability measure a on S if V converges weakly to n as T + 0c, formally,
lim T-
((s) (v)(s)
((s) ds(s),
E C(S).
(25)
The trajectory statistics measure can be simplified as an integral of delta functions, where
/ Ø(s) ((s)
1 1 [ 66g x(t)r(()))))))))))))))) dt.
ds)(s)
(26)
T
Intuitively, a trajectory is ergodic with respect to the measure æ if explores A a manner that is commensurate with T co.
Definition 5. Ergodic Cost Function [2]. Given a probability measure л on S. m-ergodic cost
function is defined E. Tx)) + R such that for any infinite trajectory x(t) Rzo + X, if
lim & (alpp.)) +0
(27)
T0000
then z(t) is ergodic, where
= U ()), T>0
(28)
as the set of all continuous trajectories defined arbitrary finite time intervals. Throughout this article we stick to X R" and - - and, we use the spectral methods and construct a metric in the Fourier space define the ergodic metric for trajectory optimization [2, 52, 1J.
Definition 6. Spectral Ergodic Cost Function. Let π be a probability measure on S. Let K" C C be the set of all integer fundamental frequencies that define the cosine Fourier basis function
(WekeT
Fu(w))
II cos
(29)
h
L;
where hu a normalizing factor (see [8]). For finite trajectory r(t): [0, + X, let Vr be the measure defined in Eq. (24). The spectral ergodic cost function is defined as
Ex(x) = Σ A ( - 2²²²
(30)
(I, Fallo - Fa(o))d((s)
Σ Ax
REK"
where V and n² the 1 Fourier decomposition modes of Vr and ps, respectively (using Eq. (26)),
and Ag = (1+ [|(||[) modes.
=22 a weight coefficient that places higher importance on lower-frequency
In particular, the spectral ergodic cost function defined for a probability measure forms a metric [8].
Alternatively, one can use the kernel MMD to construct a ergodic metric for trajectories on any generalized domains [60].
We now can formulate an (ergodic) trajectory optimization using the ergodic metric as
minimize z(())r(x))
(x(()))
(31)
e2)) vie|o,r]
subject to
hy (x) = 0,
VI E [
h (x: < 0, Vt E [0,
where by are the quality constraints e.g., à f(z u) and ha the inequality constraints.
20

---

E Stein variational ergodic search
E.1 Likelihood of ergodicity and stein variations
Stein Variational Ergodic Search (SVES) [2] recasts the trajectory optimization problem as a (varia- tional) inference problem. They begin by introducing binary indicator function
0 T + {0. 1),
which flags whether given trajectory satisfies desired optimality condition. Let æ be a probability S. The likelihood that trajectory ær exhibits the desired
measure defined on the exploration domain ergodic behavior through an ergodic cost function & (2) is given by
p(O x)) exp (=))
where μ > O is scaling parameter that governs the influence of the ergodic cost.
By combining this likelihood with prior p(x) over the space of trajectories via Bayes' rule, one can obtain the posterior p(x O). This posterior is approximated using Stein Variational Gradient Descent (SVGD). this framework, the update direction that minimizes the Kullback-Leibe divergence (from equation 8) is given by
お (.) = Exng ((,, (V logp(x) - (x)) + Vzk(z,))
(32)
where (xx,)) a positive definite kernel on the space of trajectories. This vector field steers each trajectory sample toward regions of lower ergodic cost while ensuring diversity across samples.
E.2 Ergodic stein variational trajectory optimization
For computational tractability, discretize trajectories. Let trajectory be represented
x [o 31,
-1)
with 24 E X. An empirical distribution over discrete trajectory samples is given by
i- 8xi.
Typically one assumes a Gaussian prior of the form
p(x) = (
where è is typically chosen interpolation between specified boundary conditions.
Constraints on the trajectory can be integrated into the cost likelihood function in the following form
(x) (x) + p(x) + e: hy (x)² + C2 max(0, hx()))
(33)
where C1, C2 are (positive) Lagrange multipliers that form an inner product with equality and inequality functions f1, f2, and =(x) R is any additional penalty terms on the trajectory x. Now, the ergodic Stein variational update step using Jn is given by
N
+;(-)
=
1 N
Σ [xx, (V (ogp()))())))) - VV V=k(x,,)) )]
(34)
using the kernel a ixx, This procedure exploits the repulsive characteristics of the kernel to prevent sample (or mode) collapse, thereby ensuring that the resulting trajectories remain diverse well-dispersed and effective for exploration. preventing mode collapse fundamentally depends on selecting appropriate and effective kernel. The algorithm outlining this procedure is as follows:
crucial to note that maintaining diversity and
*One can use augmented Lagrangian or any interior point method to guarantee constraint satisfiable (although in practice this may not be required in most cases). Alternatively one may choose integrate constraints as described in [61]
21

---

Algorithm 2 Stein Variational Ergodic Trajectory Opt.
1: input: measure и, domain S, map g: X' + S, cost Jr, prior p(r), kernel k(x,,)) :), step size
iteration r = initial trajectory samples {-²) termination condition y
2: while Some convergence criterion do
3: for all samples in parallel do
4: 2++1 f 2 + (x,))
5: end for
6:
/K= --++1
7: end while
8: return: {am)) arg max; (xp(-()a(x1))
E.3 Ergodic stein variational control
This approach can easily be extended to the control domain by optimizing over sequences of control inputs rather than trajectories directly. Let the control sequence be represented as
u [2o, U1 -1l,
and let the the system evolves according to the discrete-ting dynamics
Z++1 (Te, ue),
given To-
We define an empirical distribution over control sequences by
N
q
N
and assume prior p(u) on the control sequences. With a kernel k(u, defined on the control space, the SVGD update for controls is expressed
N 0,(-) = N Σ [k (V logp((() - uV V mn)) + V Vuk(u ))
(35)
where Jn (x//u//. 20) represents the ergodic (with constraints) crated under the control sequence u, starting from the initial state zo. This control formulation is well-suited for online applications, where the first control action is executed and the sequence is re-optimized at each time step to adapt to new information.
incurred by the trajectory gen-
The algorithm summarizes the approach for control optimization within a model- control framework is follows:
Algorithm 3 Stein Variational Ergodic Control
1: input: initial state To. time horizon T, measure domain W, map 9: X' S, prior p(u), kernel k(u,,), :), step size e, prior control samples {u.) termination condition y
2: k- O
3: while Some convergence criterion do for all samples in parallel do
4:
5: u++1 + ut + c (),)
6: end for
7: P- +
8: end while
9: return: {uf) ( =-= arg max, exp
apply ug to robot
11: /* shift controls +/
for all samples in parallel do
13: 0;,0TT-2 + 14,1:7-1
end for
15: /* sample state and return to input */
22

---

F
Convergence of SVGD in the Ergodic Search Setting
The work in [2] shows that the general convergence results for SVGD from [30] remain valid in the ergodic search setting when using an RBF kernel. We now generalize this convergence result to any differentiable kernel.
Consider a discretized ergodic search on a bounded state space X I and normalized workspace W = [0, 1 with T time points. Without loss of generality set X' [0, 1]", and thus SVGD acts on the space X = [o, 1] T""_ We fix a mapping g: X' + X. Moreover, we work in the population limit of infinitely many initial samples from the prior.
Let & be the spectral ergodic cost function with respect to a measure æ on W, and let p be the prior on X², Then the posterior
= p(x| O) x p(0 (012))pt p(x)
can be written in the form
~~ e (z) logp(z)).
(27)
Define the potential function
V(x) = (x) log
(28)
as is standard in SVGD.
Furthermore, let k: X x X2 - IR be positive definite kernel with associated RKHS H. Conver- gence of SVGD is then characterized by the kernel Stein discrepancy (KSD). In particular, for any measure q, its discrepancy to is
KSD,(d) = 11 (w)l
(29)
where og HHT (noting X C T is given by
6" (4(2) = Ex [,,,, (V, logp(r) (z)( + Vak(z,))
(30)
In particular, (g) coincides with the norm of the SVGD update gradient of in (32). Here define SVGD in the population limit the same way as the finite-sample case. The population gradient iteration is of 6² (q,), where 9, is the pushforward of 9--1 under the map U, X² X²T
U,(2)
+ 1
(31)
with step size > 0 and initial distribution go = p.
Moreover, the convergence results in [30] rest on three assumptions on the kernel k, the potential V. and the updates U,.. Specifically, there exist constants B, C,C,M. such that:
(AI) [|{(,, 1 S B and || V/,,( ) S
(A2) The Hessian Hy of V in (28) is well-defined with operator norm | llop M. (A3) For all KSD,, S C.
Now can formally state the generalized convergence guarantee from [2] follows:
Theorem 2. Let X = [o. 1]", let n p(z O) be the target with potential V from (28), and assume p is smooth on XT. Let ls be positive kernel on X" with RKHS H. Then there exists step size e < S, where depends on B, C,M, such that
KT(pl)n)
Σ KSD,(()) S
(32)
cer
where e, a constant depending on e, M, and B.
Proof. This result is special case of [30, Corollary 6]. and in order to prove this result, we must show that assumptions (A1), (A2), (A3) above hold. First, (A1) holds as long as the kernel k is differentiable, and XT is bounded domain. Next, we note that for a discrete path x = (zo....,.0 -T). the spectral ergodic cost fucntion has the form
T-1 Fa(o(Zx)) - J. Fa(())d((w))
(x) = Σ As
Σ 1=
(36)
BEAT
23

---

which is smooth since FR. from Eq. (29) is smooth. Because the prior is also smooth, the potential V is smooth, and the Hessian is well-defined. Furthermore, the Hessian bounded since X" is bounded domain, so (42) is satisfied. Finally, since (A1) and (A2) is satisfied, it suffices to show that
sup X
|x]| dq-(x) < oc,
(37)
to show (43), from the discussion in [30, Section 5]. However, this is satisfied since is a probability measure and W is bounded domain, (A3) is satisfied.
G
Kernels
G.1 Radial basis function (RBF) kernel
The Radial Basis Function (RBF) kernel [62], also known as the Gaussian kernel, one of the most widely used kernels in machine learning and other applications due to its strong theoretical properties and practical performance. Given two inputs z, E Rd (In the case of our trajectories, [ UT| where x; E Ik", flatten them into a vector in I T"), the RBF kernel is defined as
KREF(I, = exp ||
11
(38)
2h
where > the kernel bandwidth, and is chosen using the median heuristic. This choice kernel corresponds to inner product in an infiniteddimensional reproducing kernel Hilbert space (RKHS), making it a universal approximator under mild conditions on h.
G.2 Markov RBF kernel
Given state trajectories
I1, .1....,, æT], [1 1y...... . yrl,
we define Markov kernel by decomposing similarity along the Markov chain:
(x,(x,)) y)
Σ k(z1, ve) +
Σ (L,))
k( Vi)
where each l: a some (positive definite) kernel and 9 is a graph of connected points separated by time i.e., chosen to encode the Markov-structure along time (typically the nearest-neighbour edges t ++ ++ 1).
Choice of the temporal graph G. For sparse Markov kernel one usually takes G.n. {(t, t 1) so that only nearest-neighhot time indices interact. This nearest- neightt- choice was the one employed in the experiment described in section 7.2.
In the complete-graph variant used in the experiments in sections 7.1 & 7.3.
Gommplete
{ (t)
15 s)
which connects every ordered pair of distinct time indices. Substituting this into the definition gives
complete
(x,y)
ΣΣ M(r.
and dividing the double sum by 2 simply rescales the kernel without affecting positiveedefiiiieness or SVGD updates.
A typical choice for the base kernel is the radial basis function (RBF) with bandwidth h set by, the median heuristic in our experiments. This construction yields an expressive yet efficient measure of trajectory similarity that honors the one-step dependence characteristic of Markov processes.
24

---

G.3 Global alignment kernel
Let x [.i,...., Fnl and y ........ Vm be two trajectories (of possibly different lengths) in Rd. Traditional vector kernels (e.g., Gaussian polynomial) cannot directly handle variable-length sequences nor account for temporal distortions. Dynamic alignment methods such as Dynamic Time Warping (DTW) (see Appendix G.4) overcome this by finding an optimal alignment, but do not yield positive-definite kernels in general.
The Global Alignment (GA) kernel [63] addresses these issues by summing over all possible align- ments between x and y. Formally, let
{
1 S S..
A(x,))
((i..i))...,(i...ii)) (iz,
= S...... jj==
(i+-1 ix) E (0, (it+1 jk) E (0.1)
be the of all monotonic alignments between the indices of and y. Given local positive-definite kernel (often called static kernel)
k:X
R.
the GA kernel defined as
0 (x,y)
Σ
II M(1.8;).
xEA(z.v) (i.))
Intuitively, (x,, y) aggregates the similarity contributions of every alignment, acting "soft-max" over DTW-like scores, In our experiments we chose the RBF kernel as the static kernel.
Positive definiteness. Under mild conditions on k (for instance, if both k and k/(1 k) are positive definite), K itself Mercer kernel.
Efficient computation.
Although AA(x, y) grows exponentially, k3a (x,y) admits simple dy- namic programming algorithm in (nmdd) time. Define
1 O,
i i= j == =0 0, i= 0 <
Mrj [
<i,
k((),1))) (Mirs + Mi,-1 M1-1-j1),
i 1.12..
Then
K (<.,)) v) Mn.m
This quadratic-time computation matches the complexity of standard DTW while incorporating richer alignment information. And this can be further reduced to wall-clock time complexity of ((md) (without loss of generality and m) on GPU that can accommodate the required number of threads.
G.4 Kernelized dynamic time warping
Let xetx== [21, [,, and y [Vi...., yml ] be two trajectories (of possibly different lengths) in Te.. Traditional vector kernels cannot handle temporal misalignments and variable lengths inherent in time-series comparison. Dynamic Time Warping (DTW) addresses this by computing an optimal alignment path n that minimizes the cumulative local distance:
(xTw(x,y) min TEA((xy)
Σ [T)(1) 2=
Kernelization of DTW (KDTW). [64] To turn DTW into Mercer kernel many but very similar ideas have been used, we will state such ideas. One approach exponentiate the optimal cost, yielding the dynamic time alignment kernel:
(VTWI(X,)) = exp
min
Σ
(m.10) (0) - VTa(e (1)
212
TE A(x()
25

---

An alternative, inspired by convolution kernels [65], is to exponentiate local distances first and then maximize over alignments:
x
(x.y) = max reA(x.) -
exp
26 [r)(()
Neither construction guarantees differentiability.
We use PTWI in our experiments and h is chosen using the median heuristic at every iteration of SVGD SV-CMA-ES. We approximated the gradient of the DDTI kernel using a centered finite- difference scheme in our SVGD experiments. Although the KDTW kernels are not differentiable in the classical sense, we observed no adverse effects in our experiments. Nevertheless, these theoretical issues warrant careful attention in future work.
Computational complexity. Both DTWI and DDTW2 admit O(nmd) Senamic-programmen algorithms, replacing the usual min operations of DTW with exponentiation and max (or further min) steps, while preserving quadratic time complexity. And this be further reduced wall-clock time complexity of O(md) (without loss of generality on n and m) on a GPU that can accommodate the required number of threads.
G.S Signature kernel
Definition 7. Signature (transform) of a path (or trajectory) [20]. Let V be a Banach space and define the tensor algebra of formal power series
T((V))
vok
k20
which is equipped with the natural operations induced by the tensor product and has the unit element (1, O, .....).
Now, consider a continuous path x: : - on a compact interval T C T with finite p- p-variation (for some < 2). For any subinterval [s, C T, the signature of over [s, f) is defined as the sequence
xp(x),,= = (1, Jacuzes dæ(u)),...
dx(u)) 2 - dx(ux),... ...) T((V)).
(39)
This collection of iterated integrals captures the essential features of the path and is invariant under reparameterization.
Recall, the signature of a trajectory is injective (for non-tree-like paths) and, remains unchanged under any reparametrization. This invariance effectively filters out the complex, infinite-dimensional group symmetries. Moreover, the collection of linear functionals defined on the signature not only forms algebra under pointwise multiplication but also has the power to separate points [66]. Consequently, by the Crone-Weiersrran bounded variation, the set of such linear functionals is dense in the space of continuous real-valued functions on C [67]. These properties together establish the signature as an ideal feature map for representing trajectories in ergodic search.
theorem, for any compact set C continuous paths with
For path x of finite p- variation with p > 1 the signature terms decay factorially [67]
U/ <aa < <ust
| 1
p.le.
dr(u1) ) 8 ... @ da dr(ux)
k!
188
where ||>:|p,|s.: j is the p-ariiation of the path on t). Therefore, one can approximate signature kernel reasonably well with the truncated signature transform of level I. E N.
((),(())) (1. f do(ui),
dæ(wi) 8 ... @ de(ur))
E
Vosk
k=U
(40)
where denotes the direct sum, and [21] and [22] provide efficient means to compute the trun- cated signature using Dynamic Programming and approximate using Random Fourier Features, respectively.
26

---

Definition 8. Signature Kernel Let I = u²] and J = [u. be two compact intervals. and consider paths (I, V) and y C¹ (J.V), where V is a Banach space. The signature kernel k:* (z. y) T x J - R is defined by
kad (z, u) ((((x)ss) sp())))
(41)
where p(x), and ply)e denote the signatures of x and y over the intervals [u, and [v, t). respec- rively.
Utilizing the kernel trick introduced [20], one can efficiently compute the full (untruncated) signature kernel.
Theorem 3. Let I = [u, u') and = [u, be compact intervals, and let x E C (I,V) and y E (J.V). Then the signature kernel Kx,y satisfies the following linear, second-order hyperbolic partial differential equation:
02 Ös (x,y) (i(s), ((t)) k2,(
subject the boundary conditions
ka (z, v) =1 V1
(g -1 4s
The resulting PDE formulation the signature kernel enables its efficient computation using any standard numerical scheme, such finite difference, finite element, or other suitable methods. An important point to note is the assumptions on Theorem 3, will be satisfied for any reasonable discretizations of the trajectories and the domain over which the PDE solved.
One must take extra cation they wish to use the signature kernel SV-CMA-ES, since the trajectories produced in initial few iterations of SV-CMA-ES may not be very well behaved, can produce very rough trajectories, breaking the assumptions of Theorem 3.
G.6 Median heuristic
The median heuristic [27] sets the bandwidth
med (f - xılla)ies
h
log N
using the empirical median of all pairwise distances among particles (divided by log N) to provide a robust, data-driven scale. We adopt
med (1x})
log N
which yields slightly improved performance (in terms of computational speed) in our experiments. Additionally, we employed the median heuristic for bandwidth selection because it is straightforward, computationally light, and requires no manual tuning, making our method more accessible users. We recommend referring to [27] for more (theoretical) guarantees, advantages and details on the median heuristic.
H Measuring trajectory diversity
One can interpret the kernel k(x, y) as similarity measure between two trajectories x and In partic- ular, for a normalized kernel function (i.e., k(x, x) = 1), k(x, y) +1 as x x-y, A( y) + 0 ||x - y + by forming the Gram matrix K E N K,₃ = k(x', x)). K₁ = 1. and computing its deter- minant. Notably, det(K) + 0 if x² + x" for any i & j. oo for all A thereby providing a principled measure of particle diversity.
Given collection of trajectories (x' J Jizi one may quantify their diversity
det(K) + 1 as | -
Throughout the experiments we shall use the following kernel for measuring diversity,
deontimoous Frochet (x, y)
Frochet (aroche(xx, y) = exp
(42)
2h2
27

---

where deoatimnous Frecher (-:-) is the (Continuous) Frechet distance (see Appendix 1) and h = 0.1. We denote the Gram matrix of this kemel with K-
One may choose any finite-valued kernel measure diversity, chose k F-eehet simply due the fact that has not been used in experiments to benchmark the signature kernel.
I Fréchet distance
The Fréchet distance [68] is a classical similarity measure between trajectories that respects both spatial geometry and the ordering of points along each trajectory. Intuitively, the minimum leash-length necessary for person and dog to walk along two continuous paths, each at its own varying speed, without backtracking.
Definition 9 (Continuous Fréchet Distance). Let f.9: [0. (X.d) be continuous curves in metric space. reparametcrization a continuous, nondecreasing, surjection a: [0, - [0, satisfying
(0)) = 0,
a(1)= =
(and similarly for 3). The continuous Fréchet distance is
dContinuous Frecher Frecher(JJg))
inf 0.8 reparam
max ((((((()))(((()) tE[o.
In our case with discretized trajectories in Rd
[2 , Fnl, y ..
we use the discrete Fréchet distance.
Definition 10 (Discrete Fréchet Distance). A coupling of x and is pair of index sequences and 1, T(K) = m, and each of a, T is non- non-decreasing with increments at most 1. The discrete Fréchet distance is
(r( T(K)) such that o(1) = o(K) n, T(1)
d'recher (x,y)
min max 0,t couplings k= I=I...,
|| [oo()) Vr(x))l-
Computational complexity
The continuous Fréchet distance can be computed in
(d log(nm)),
time [69, 70J. Here-and unlike most of the literature that treats as a hidden constant-d varies in our setting and so is made explicit in the O-)) notation.
Properties
Metric Properties. The continuous Fréchet distance is only pseudometric on the space parameterized curves distiint parameterizations
the same geometric trace satisfy dCmminuous (f,g) O-bbut it induces a true metric on the quotient of curves under reparameterization. In contrast, the discrete Fréchet distance drreshet on point-sequences satisfies all four metric axioms (non-negativity, symmetry, identity of indiscernibles, and the triangle inequality) and therefore a bona fide metric on the set of discrete trajectories
Stability. If two curves f' and g' satisfy ||f -' S and l|g g' oo S e, then [df.g) d(f', g')l < E.
Order-sensitivity. Unlike the Hausdorff distance, the Fréchet distance respects the natural parameterization of trajectories, making it especially powerful for comparing time-indexed
data.
Because of these advantages, the Fréchet distance has become comerstone in shape matching, motion analysis, and trajectory mining.
Additional results
J.1 Multiscale constrained exploration
28

---

RBF Kernel
(b) Global Alignment Kernel
(c) Signature kernel
Figure 9: Trajectories generated using different kernels in the multiscale constrained exploration experiment comparing kernels section 7.1.I.
(a) SVGD 0.05)
(b) SV-CMA-ES (o,, 0.2)
Figure 10: Trajectories generated using SVGD and SV-CMA-ES in the multiscale constrained
exploration experiment comparing the SVGD and SV-CMA-ES in section 7.1.2.
29

---

Table 2: Aggregated metries across 10 priors for each kernel for the Multiscale constrained exploration
Kernel
Metric
Mean
StdDev
Min
Max
RBF
Best trajectory cost
0.4203
0.0297
0.3742
0.4833
Average trajectory cost
0.5152
0.0399
0,4190
0.5666
Mean ergodic cost
0.1816
0.0194
0,1308
0.1979
Trajectory diversity
2.2052
0.7413
1,0172
3.5636
Signature Kernel
Best trajectory
0.4145
0.0564
0.3423
0.5479
Average trajectory cost
0.4916
0.0541
0.4065
0.5995
Mean ergodic cost
0.1677
0.0234
0.1295
0.2120
Trajectory diversity
3.3854
1.2080
1.8504
5.4397
Kernelized DTW
Best trajectory cost
0.4249
0.0228
0.3925
0.4771
Average trajectory cost
0.5275
0.0457
0.4320
0.6185
Mean ergodic cost
0.1865
0.0229
0.1368
0.2215
Trajectory diversity
1.9260
0.6717
0.8498
3,3909
Markov RBF
Best trajectory cost
0.5270
0.0496
0.4421
0.6267
Average trajectory cost
0.5346
0.0488
0,4501
0.6326
Mean ergodic cost
0.1762
0.0187
0.1519
0.2122
Trajectory diversity
0.00005
0.00002
0.00003
0.00009
Global Alignment Kernel
Best trajectory cost
0.4767
0.0626
0.3763
0.6247
Average trajectory cost
0.5732
0.0599
0.4912
0.6928
Mean ergodic cost
0.1740
0.0299
0.1200
0.2274
Trajectory diversity
2.8324
1.3845
0.7104
5.5709
30

---

J.2 3D Coverage using the Crazyflie drone
(a) SVGD (c 0.01)
(b) SV-CMA-ESS (a,, 0.1)
Figure 11: Top down projection of trajectories generated using SVGD and SV-CMA-ES the coverage using the Crazyflie drone experiment comparing the SVGD and SV-CMA-ES in section 7,2.2.
31

---

Table 3: Aggregated metrics across 10 priors for each kernel for the 3D coverage using the crazyflie drone
Kernel
Metric
Mean
StdDev
Min
Max
RBF
Best trajectory cost
1.1923
0.0415
1.1105
1.2498
Average trajectory cost
1.3216
0.0213
1.2884
1.3700
Mean ergodic cost
0.7118
0.0172
0.6868
0,7380
Trajectory diversity
7.0293
1,0004
5.2130
8.4017
Signature Kernel
Best trajectory cost
1.0071
0.0323
0.9503
1.0509
Average trajectory cost
1.1283
0.0171
1.0923
1.1596
Mean ergodic cost
0.5574
0.0143
0.5319
0.5798
Trajectory diversity
10.3330
1.0336
8.4339
11.4770
Kernelized DTW
Best trajectory cost
1.0140
0.0247
0.9698
1.0588
Average trajectory cost
1.1413
0,0183
1,1026
1.1695
Mean ergodic cost
0.5702
0.0121
0.5412
0.5866
Trajectory diversity
8.9372
1,6642
6.0877
12.2700
Markov RBF
Best trajectory cost
1.3248
0.0440
1.2547
1.4031
Average trajectory cost
1.4934
0.0310
1.4592
1.5560
Mean ergodic cost
0,8445
0.0257
0.8037
0.8860
Trajectory diversity
5.1746
1.3442
2.6743
7.4214
Global Alignment Kernel
Best trajectory cost
1.1251
0.0347
1.0638
1,1850
Average trajectory
1.2507
0.0209
1.2183
1.2833
Mean ergodic cost
0.6489
0.0126
0.6245
0.6741
Trajectory diversity
7.3018
1.7350
4,4249
9,9842
J.3
Stein variational model-predictive control
32

---

Table 4: Aggregated metrics from Table
1 across all 10 priors.
Kernel
Metric
Mean
StdDev
Min
Max
RBF
Total best trajectory cost
382.347
14.278
377.832
422.982
Total average trajectory cost
574.443
35.744
563.140
676.174
Total ergodic cost of best trajectory
268.319
19.699
262.090
324.385
Total trajectory diversity
49.986
4.655
48.514
63.233
Signature
Total best trajectory cost
360.060
23.569
340.256
420.901
Kernel
Total average trajectory cost
592.608
46.984
552.196
719.126
Total ergodic cost best trajectory
244.615
24.056
225.835
309.706
Total trajectory diversity
566.583
153.891
360.379
925.329
Markov
Total best trajectory cost
476.244
23,475
468.820
543.055
RBF
Total average trajectory cost
775.057
34.687
764.088
873.777
Total ergodic cost of best trajectory
328.503
27.263
319.882
406.095
Total trajectory diversity
3795.151
679.803
2050.583
4080.356
Kernelized
Total best trajectory cost
364.486
13.236
347.587
383.277
DTW
Total average trajectory cost
561.385
14.056
542.033
581.960
Total ergodic cost of best trajectory
256.216
13.246
240.409
276.713
Total trajectory diversity
68.739
12.963
55.162
98.367
K Implementational and experimental deatils
In this section we provide the full Implementational and experimental details.
K.1
Implementational details
Elite sample selection and log-rank weights for SV-CMA-ES In experiments rank the m samples by selecting the top
as the elite sub- poopoooulaiinn we assign positive log rank recombination weights
log(A + 0.5) logr r
Wi,k
I,...,A, X,
log(A> + 0.5) - log
and set U/,A ( for &s > X. ("Active" negative weights [39] are disabled in the present implementa- tion.)
Enhancing the signature kernel's performance A common enhancement is to first embed each trajectory into (possibly infnite-dimensiona)) feature space via static feature map Pstatic Rdd
H. associated, for example, with radial basis or Matérn kernel. Concretely, one defines
Patatic (x) = (x)))
Empirical studies demonstrate that this pre-lifting step substantially improves the performance of the signature kernel. Therefore, the ergodic Stein variational update step in the state space using the
33

---

signature kernel and a static kernel is given by
07(-) N
k (pstattcex)) Pstatio (-)) log
nV x v V (Fstatio(x); Pstatic()))]- (43)
An analogous update applies in the control space. Throughout this paper, emploied the feature map corresponding to the radial basis kernel (RBF) (see Appendix G).
Hardware used All experiments were run on the computer with:
1. Operating System: Ubuntu 20.04.6 LTS
2. CPU: AMD Ryzen 9 5900HS
3. RAM: GB
4. GPU: NVIDIA GeForce RTX 3060
5. CUDA Version: 12.2
Assets used The codebases provided by the following works [2, 22, 20] and the GitHub repository ptttps://gtthn www/kkdlr/ http///dd//sffftttete ments or proved essential as implementation references.
either directly incorporated into experi-
K.2
Experimental details
K.2.1 Multiscale constrained exploration
We consider the two-dimensional spatial domain S = [0, 100] [0, 100] m. equipped with the
uniform measure л. To improve numerical conditioning. we introduce the affine mapping g: S + Trajectories of dimension = 2 and length T 100 with time step of At = 1 are generated via Algorithm described sections 3.1 & E.2. maximum kmax = of basis functions per dimension is employed for constructing the ergodic metric (see Appendix D
[0, x [0, 1], g(x)
Definition 6). The cost functional specified as
T
(x) = Ex) + (xs
+
15 Fa+1
(=0
0.1 ||10 xinit 1 0.1 ØT Zfinal 1 0.01 Coba (x), (44)
where the obstacle penalty is defined by (x) max (0. || - xeel2 r), with v. and denoting the obstacle center and radius, respectively. The initial and final states, Xinit and Xfiall are prescribed on the boundary of S.
'This normalization does not affect the optimization, since the ergodic metric is defined in the Fourier spectral domain any periodic domain.
34

---

Table 5: Full Hyperparameter overview of SV-CMA-ES in the multiscale constrained exploration
example
Hyperparameter
Value
m
32
A
6
0,
(0)
0.1 (initial self adaption rate)
O
(Aeft + 2)/(M + Acff + 5), where M is problem dimension,
i.e, for a trajectory with T time steps and in d dimensions, M = Td
do
1 max (0. /
Aa-11 M+I
- 1) + Cke
Oke
(4 + (M +4+ 2ett MM)
2/ ((M 1.3)² + Acft)
on
min 1 a1, 2( - 2 + 1/>att/ + +2
2 /2)
ax
k(:-))
RBF kernel
K.2.2
3D Coverage using the Crazyflic drone
We consider a
domain S = [0, 3) x [0, 3] x [0.5, 1.5], equipped with the uniform
measure Trajectories computed via Algorithm described in section 3.1 & E.2, using cost functional analogous to that the multiscale forest example:
(x)x) (x) + (x)
Σ
15 r++1 - xx|12
1=0
+ 1 -0 Xmit 12 +.1 FT
2 0.01 Cobs (x). (45)
Zfinal
The trajectory length is set to T = 150 with time step of At = 0.1s. A maximum kmax 8 of basis functions per dimension is employed for constructing the ergodic metric (see Appendix Definition 6).
Table 6: Full Hyperparameter overview of SV-CMA-ES
Hyperparameter Value
m
32
A
6
(0)
0.2 (initial self adaption rate)
Qg
(Ner + 2)/(M Aet + 5), where M problem dimension.
i., for a trajectory with time steps and in d dimensions, Td
1 max(0, / ' - 1) + ar
dz
0e
(4 + XeneM) + +4+2 2
2/ (M 1.3)2 + Adel)
Q1
min 1-aaj a1, 2(x 1/>ett/ + +2² + 2 12)
ax
k(.,))
RBF kernel
35

---

K.2.3 Stein variational model-predictive control
We consider coverage over quad-modal target distribution
x (x)
(us, 12
comprising four Gaussian components
1 (0.2, 0.2).
01 300
122 (0.85, 0.85),
V /300
T (0.23, 0.75),
03
V V300'
1/4 (0.75, 0.2),
/300*
on the unit-square domain S= 0, 1] [0. 1]. The agent obeys single-integrator dynamics I1+1 Tt + Al ur, with A/ = 0.1. Trajectory planning carried out over horizon of T== time steps, using N = 20 control-sequence samples per iteration. Each planning loop is executed for up to 200 iterations.
A maximum kmax = 10 of basis functions per dimension is employed for constructing the ergodic metric (see Appendix D Definition 6). Ten obstacles are placed uniformly random in S; each obstacle moves at the same rate At, with velocity directions sampled from (N0, .²) where a 0.01.
The cost functional for control sequence (uo..... wT-1) is defined
(u) = (u) + cx( + 0.01 ||u 2 + 0.001
>
zz =r.12 100 Caba (x).
(46)
t=0
with scaling parameter 10. The initial control prior is taken as p(u) -N 0.01 1).
36

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The claim made in the abstract and introduction are supported in section 7
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Limitations are discussed in section 8.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
37

---

Justification:
Look at Appendix F.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes].
Justification:
The exact experimental setup is are provides in sections 7 & Appendix K.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
38

---

Answer: [No]
Justification: We extended the code used from the paper [2]. The work in this paper is closed source and therefore we did not open our code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips. ec/
wwooooooliioo ooloollcyyy icyy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips. cc/ mublic/guidess Caleee//ooSSubbiesce lebbbmisssiins ssisnsoliccyyyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [NA].
Justification: This paper does not introduce machine learning based algorithm.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We do provide statistical data section Although no p-test is performed.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
39

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: This is provided in section K.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https http://neerippppe cc htcpppulic//tthccaiiieeeeeceeddd
Answer: [Yes].
Justification: violated.
We acknowledge the code of ethics and can guarantee that none of them were
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA].
Justification:
The social impact this work not quantifiable moment.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
40

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA].
Justification: The paper poses no such risk. The paper builds on a trajectory optimization algorithm.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: They have been acknowledged in section K.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
41

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets, but will provide assets soon, and will
be documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA].
Justification:
The paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA].
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
42

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA].
Justification: LLMs were used only for improving the english used in the paper and, writing and debugging snippets code. They were not used for any core method development.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https //tp://nee...p// for what should or should be described.
43


Reviews:
[{"review_id": "2KmO73KOKS", "reviewer": "Reviewer_KSg3", "summary": "This paper improves Stein Variational Ergodic Search (SVES) for robotic exploration by making trajectories more diverse and lowering ergodic cost. It does this by using a signature kernel and a gradient-free evolution strategy (SV-CMA-ES). Experiments on various tasks demonstrate these improvements.", "questions": "1. Could you expand the related work to compare with other ergodic planners or diversity-focused methods outside SVES? \n2. Could comparisons be made to other non-SVES baselines?\n3. Is there any hardware or real robot demo?\n4. Could you provide more details on how sensitive results are to kernel parameters?", "limitations": "Yes, the authors acknowledge key technical limitations, including hardware scalability, computational cost for large particle sets, and practical issues like the signature kernel pushing trajectories into obstacles.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Clear technical improvement on SVES by combining signature kernels and SV-CMA-ES.\n2. Experiments support claims with consistent gains in trajectory diversity and lower ergodic cost.\n3. Well-structured and clearly presented paper.\n\nWeaknesses:\n1. Could include more related work to better position the contribution.\n2. The main novelty seems to be the combination of known components, not new core methods.\n3. Missing baselines from recent literature for broader comparison.", "was_revised": true, "final_justification": "The authors address my questions reasonably. However, my my rating remains the same.", "rebuttal": "# 1. Could you expand the related work to compare with other ergodic planners or diversity-focused methods outside SVES?\n\nWe thank the reviewer for pointing out the need for including more related work to better position the contribution.\n\nClassical ergodic planners focus on steering a single trajectory (or a centrally controlled fleet) to match a target spatial distribution (Section 1 and Appendix A). In contrast, our SVES‐diversification methods generate a set of distinct or diverse ergodic trajectories in parallel, leveraging repulsive interactions or signature‐based seeding to ensure both high coverage and trajectory diversity in a unified optimization.\n\nOutside of the ergodic literature, the community has developed a rich variety of quality‐diversity and diversity‐maximization techniques such as MAP-Elites [1], Novelty Search [2] and Determinantal point processes [3]. While these methods successfully assemble static repertoires of diverse solutions, they do not directly optimize for ergodic coverage over time.  Our contribution is to bring diversity‐aware techniques into the ergodic planning domain.\n\nAt a high level, MAP‑Elites keeps the single best example for each kind of behavior so you end up with a set of good but different options. Novelty Search a evolutionary-computation paradigm, abandons traditional fitness objectives, instead it rewards behavioral divergence; it rewards experiments that lead to brand‑new behaviors, so the search naturally promoting exploration.  Determinantal point processes (DPP) give a straightforward way to pick a batch of items that are both high‑quality and as different from one another as possible.\n\n\n\n# 2. Could comparisons be made to other non-SVES baselines?\nPrior work suggests that the non-convex form of ergodic search methods are capable of identifying multiple optimal trajectories through variations in initial conditions (see reference [9] from our paper). Hence, the main focus of our work is to produce a set of diverse locally optimal trajectories with effective mode coverage. As long as conditions for optimality are the same across both the SVES based and the non-SVES algorithms both should converge to same or similarly optimal trajectories since SVES guarantees convergence (see [2] from our paper).\n\n# 3. Is there any hardware or real robot demo?\nThank you for highlighting the importance of real‐world validation. In fact, we have conducted hardware experiments with the Crazyflie drone. Figure 1 in the paper illustrates the paths taken by the Crazyflie drone in a real environment, and the quantitative results are reported in Section 7.2. To further underscore these real‐robot findings, additional images and videos can be added after this rebuttal.\n\n# 4. Could you provide more details on how sensitive results are to kernel parameters?\nWe once again appreciate the reviewer suggesting a deeper investigation into bandwidth sensitivity as this is indeed very important and not well presented in the paper.\nWe will add the following augments to the Appendix to strengthen the paper.\n\nWe employed the median heuristic for bandwidth selection because it is straightforward, computationally light, and requires no manual tuning, making our method more accessible to users. Additionally, refer to [4] for details on the guarantees provided by the median heuristic. \n\nHowever, we find that setting $h=0.1$ for all kernels in the experiment from Section 7—the signature kernel still outperforms the other kernels. We did not find the optimal bandwidth for each kernel (say by grid searching) and compare them. This limitation is acknowledged in Section 9. \n\nBut in theory, we expect the signature kernel to retain its advantage thanks to its theoretical properties (discussed in lines 154-158 \\& 768-775 of our paper), most notably, the injectivity of its feature map. For a deeper mathematical treatment, we refer the reviewer to [11,14,15,19,61,62] in our paper. To illustrate one consequence of injectivity, consider three simple paths\n1. $p_1(t) = (x_1(t),y_1(t)) = (t, 0) \\quad t \\in [0, 2\\pi]\\$\n\n2. $p_2(t) = (x_2(t),y_2(t)) = (t, sin(t)) \\quad t \\in [0, 2\\pi]$\n\n3. $p_3(t) = (x_3(t),y_3(t)) = (t, sin(t + \\pi)) \\quad t \\in [0, 2\\pi]$\n\nIt is easy to see that $k^{RBF}(p_1,p_2) = k^{RBF}(p_1,p_3)$. In contrast, because the signature transform is injective on all non–tree‑like paths, no two such trajectories share the same signature, and thus the signature kernel cleanly separates these examples. In general the kernelized DTW's and Global alignment kernel's are not feature maps are not injective as well.\n\n# References \n\n[1] Jean-Baptiste Mouret and Jeff Clune, Illuminating search spaces by mapping elites, 2015,\narXiv:1504.04909, https://arxiv.org/abs/1504.04909\n\n[2] Joel Lehman and Kenneth O. Stanley. Abandoning objectives: Evolution through the search for novelty\nalone. Evolutionary Computation, 19(2):189–223, Summer 2011. MIT Press, Cambridge, MA, USA.\nhttps://doi.org/10.1162/EVCO_a_00025\n\n[3] Alex Kulesza. Determinantal Point Processes for Machine Learning. Foundations and Trends® in Ma-\nchine Learning, 5(2–3):123–286, 2012. Now Publishers. http://dx.doi.org/10.1561/2200000044\n\n[4] Damien Garreau, Wittawat Jitkrittum, and Motonobu Kanagawa. Large sample analysis of the median\nheuristic. arXiv preprint arXiv:1707.07269, 2018. https://arxiv.org/abs/1707.07269"}, {"review_id": "KLJZmr6bbD", "reviewer": "Reviewer_GvTY", "summary": "Efficient robotic exploration requires information-aware coverage of the domain, avoiding well-understood areas, and focuses more attention on areas that are not well-understood. The paper proposes two methods to diversify Stein Variational Ergodic Search (SVES), which reduce ergodic cost and produce richer trajectory sets than SVES.", "questions": "1.\tPlease provide runtime analysis on a common hardware platform. Ideally benchmark with different number of particles, and problem dimensions.\n2.\tAre there other efficient popular heuristics other than the median heuristic for selecting bandwidth of kernels? Might be interesting to benchmark these.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The paper does a good job of explaining the problem statement, as well as explaining the underlying SVES and Signature Transform and Signature Kernel. Qualitatively, the signature kernel seems to produce richer and diverse trajectories than the other methods in Figure 2. This is further reinforced in Figure 3, where the trajectory diversity is significantly higher than the other methods, while the trajectory and ergodic costs are marginally lower. The signature kernel really stands out from methods in real-world drone tests. SV-CMA-ES shows similar costs compared to SVGD, while having higher average trajectory diversity. \n\nThe authors are honest about the limitations of their method, including dependence on hardware, and scalability of the approach. The one critique of the paper is that it claims SV-CMA-ES is a real-time method (in Contributions subsection), due to elimination of gradient computation, but no analysis is provided on this. I would have liked to see a runtime analysis of SV-CMA-ES versus other methods. The time complexity gives an idea of runtime for the signature kernel, but more quantitative evaluation is required. For the most part however, this is a complete work with supplementary results as well.\n\nSubmission is clearly written and well-organized. It seems reproducible as experimental and implementation details are provided. Results seem impactful for more efficient robotic exploration, particularly for active perception, where the robot is trying to map its environment in an unknown setting. Due to real-world tests on a drone, it seems like the method is viable for robotics practitioners, although little analysis of runtime is provided. This method improves prior works, and the SV-CMA-ES method is of considerable interest due to its ability to possibly run real-time on a drone.\nThe work is original as it tackles the problem of diversifying SVES as well as incorporating gradient-free approaches. It is also clear how the method advances previous works, showcasing both qualitative and quantitative comparisons. The work also provides new insights into the meaning of “diversity” for trajectories and improves upon different metrics, such as ergodic cost, trajectory cost and diversity. However, the motivation of integrating SV-CMA-ES is not well-formulated as the paper says “more efficient search steps”, but no quantitative evaluation is provided to back up this claim.", "was_revised": false, "final_justification": "", "rebuttal": "# 1. Please provide runtime analysis on a common hardware platform. Ideally benchmark with different number of particles, and problem dimensions?\n\nWe thank the reviewer for requesting a detailed runtime comparison. In the following, we report mean runtimes (over 10 independent runs) for SVGD and SV‑CMA‑ES on a common hardware platform, and discuss implementation caveats. \n\n## Experimental Setup\nThe SVGD implementation was fully JIT‑compiled, whereas SV‑CMA‑ES was only partially JIT‑compiled. No optimizations were applied to the signature kernel, so absolute timings may differ under a more optimized implementation. All experiments were run using the following hardware:\n\n1. **CPU**: AMD Ryzen 7 7800X3D (8 cores @ 4.2 GHz)\n2. **Memory**: 32 GB (@ 5600 MT/s)\n3.  **GPU**: NVIDIA Geforce RTX 5060 Ti\n\n\n## Results\n| Experiments               | Multiscale constrained exploration (Section 7.1) | 3D Coverage (Section 7.2) | SV‑MPC (Section 7.3)         |\n|---------------------------|--------------------------------------------------|----------------------------|-----------------------------|\n| No. of iterations         | 3000                                             | 1500                       | 100                         |\n| Problem dimensions | $T=100, \\ d=2$                            |  $T=50, \\ d=3$           | $T=20, \\ d=2$ |\n| N particles               |                                                  |                            |                             |\n|                           |                                                  | **SVGD with RBF kernel**   |                             |\n| 5                         | 2.83 ± 0.08 seconds                              | 1.45 ± 0.17 seconds        | 0.092 ± 0.0034 seconds      |\n| 10                        | 2.82 ± 0.12 seconds                              | 1.45 ± 0.11 seconds        | 0.093 ± 0.0034 seconds      |\n| 15                        | 2.90 ± 0.14 seconds                              | 1.40 ± 0.09 seconds        | 0.095 ± 0.0078 seconds      |\n| 20                        | 3.11 ± 0.17 seconds                              | 1.46 ± 0.10 seconds        | 0.090 ± 0.0043 seconds      |\n| 25                        | 2.94 ± 0.15 seconds                              | 1.44 ± 0.13 seconds        | 0.094 ± 0.0055 seconds      |\n|                           |                                                  | **SVGD with Signature kernel** |                          |\n| 5                         | 13.83 ± 0.84 seconds                             | 6.32 ± 0.47 seconds        | 0.150 ± 0.0030 seconds      |\n| 10                        | 18.33 ± 2.13 seconds                             | 6.30 ± 0.43 seconds        | 0.159 ± 0.0034 seconds      |\n| 15                        | 14.19 ± 0.41 seconds                             | 4.30 ± 1.72 seconds        | 0.157 ± 0.0033 seconds      |\n| 20                        | 14.36 ± 0.47 seconds                             | 6.13 ± 0.62 seconds        | 0.148 ± 0.0027 seconds      |\n| 25                        | 14.86 ± 0.11 seconds                             | 7.17 ± 0.46 seconds        | 0.158 ± 0.0022 seconds      |\n|                           |                                                  | **SV‑CMA‑ES with RBF kernel** |                          |\n| 5                         | 10.13 ± 0.21 seconds                             | 6.01 ± 0.23 seconds        | 0.353 ± 0.0060 seconds      |\n| 10                        | 10.34 ± 0.10 seconds                             | 6.28 ± 0.13 seconds        | 0.359 ± 0.0064 seconds      |\n| 15                        | 11.55 ± 0.62 seconds                             | 6.30 ± 0.85 seconds        | 0.360 ± 0.0049 seconds      |\n| 20                        | 12.08 ± 0.10 seconds                             | 6.90 ± 0.19 seconds        | 0.371 ± 0.0021 seconds      |\n| 25                        | 12.77 ± 0.22 seconds                             | 6.90 ± 0.94 seconds        | 0.378 ± 0.0102 seconds      |\n\n## SVGD vs SV-CMA-ES\nThe original Stein Variational Gradient Descent (SVGD) paper (see reference [10] from our paper) identifies the gradient of the log‑density as its primary computational bottleneck. Moreover, comparing the costs of SVGD and SV‑CMA‑ES is nontrivial, since both methods require evaluating gradients of the kernel and of the log‑density at each update. The key difference lies in the cost of obtaining the log‑density gradient itself. When that gradient is easy to compute or available analytically, SVGD will generally be the more efficient choice. By contrast, SV‑CMA‑ES truly shines when the log‑density is nondifferentiable or expensive to evaluate.\n\nIn practice, computing the log‑density gradient typically entails two passes through the density function (a forward and a backward pass). Evolution‐strategy (ES) updates, however, are highly parallelizable on GPUs. The ES updates require multiple parallel forward passes of the log density, and computing the weighted recombination step for each particle can all be batched on modern hardware (with say Cholesky decomposition). Thus, even though SV‑CMA‑ES performs more sampling “work,” its overhead beyond objective evaluations is often quite small. If the log density is non‑differentiable or costly to backpropagate, SV‑CMA‑ES can actually deliver faster wall‑clock convergence by avoiding gradient calls altogether.  Moreover SV-CMA-ES can have faster convergence rates when compared to SVGD with well-tuned hyperparameter (see reference [25] from our paper).\n\n# 2. Are there other efficient popular heuristics other than the median heuristic for selecting bandwidth of kernels? Might be interesting to benchmark these.\n\nWe appreciate the reviewers request to benchmark the different heuristics and an ablation study on the choice of heuristics will be included in the final revision of the paper. The reviewer may refer to the response provided to the 4th question of reviewer KSg3 for additional information beyond the heuristic. \n\nWe found that the median heuristic is an extremely popular choice for choosing the bandwidth across multiple domains and provides some strong guarantees [1]. Some of it's advantages include:\n1. **Fully data‑driven and tuning‑free**: It requires no extra cross‑validation or held‑out data: you just compute all pairwise distances and take their median.\n2. **Asymptotic consistency and normality**: Under very mild assumptions (e.g. a mix of two distributions), the empirical median of squared distances converges to the true median of a well‑defined “mixture” distance distribution, and indeed satisfies a central‐limit theorem.\n3. **Separation‑aware behavior**: If the two underlying distributions are well separated, there’s even a high‑probability gap between within‑ and between‑group distances. In that case, the median will reliably fall among the “within‑group” distances, yielding a bandwidth tuned to the scale of each cluster rather than to the far‑apart intercluster distances.\n4. **Keeps the kernel “in the sweet spot”:** By construction, it avoids two degenerate extremes; too small a bandwidth or too big a bandwidth. The median sits roughly in the “middle” of the distance spectrum, so you capture meaningful variation without blowing up or collapsing the Gram matrix.\n\n\nAdditionally, we employed the median heuristic for bandwidth selection because it is straightforward, computationally\nlight, and requires no manual tuning, making our method more accessible to users. \n\nA computationally light heuristic is extremely critical, as it has to be used in very iteration of SVGD or SV-CMA-ES. Although several computationally cheap heuristics (like the mean heuristic, Silverman’s rule‑of‑thumb, and Scott’s rule; see the table below) are available they do not in general enjoy non‑asymptotic or distribution‑free theoretical guarantees.\n\n| Heuristic                                  | Computational Cost     | Manual Tuning |\n|--------------------------------------------|------------------------|---------------|\n| Mean heuristic                             | Low                    | Low           |\n| Silverman’s rule‑of‑thumb                  | Low                    | Low           |\n| Scott’s rule                               | Low                    | Low           |\n| Median heuristic                           | Low to Moderate        | Low           |\n| Quantile‑based heuristic                   | Moderate               | Moderate      |\n| Sheather–Jones direct plug‑in              | Moderate               | Moderate      |\n| k‑nearest‑neighbor bandwidths              | Moderate               | Moderate      |\n| Penalized Comparison to Overfitting (PCO)  | Moderate               | High          |\n| Least‑squares cross‑validation (LSCV)      | High                   | High          |\n| Likelihood cross‑validation                 | High                   | High          |\n| Partitioned cross‑validation               | High                   | High          |\n\n# References \n[1] Damien Garreau, Wittawat Jitkrittum, and Motonobu Kanagawa. Large sample analysis of the median\nheuristic. arXiv preprint arXiv:1707.07269, 2018. https://arxiv.org/abs/1707.07269"}, {"review_id": "E94ZTPoixr", "reviewer": "Reviewer_cUiH", "summary": "Efficiently exploring a bounded domain within a finite amount of time is a challenging and central problem in robotics. Ergodic exploration converts exploration into a coverage problem, optimizing the time-average spatial distribution of the robot against an expected information measure. While this can handle multi-modal landscapes, in practice, the non-convex nature of the objective makes this difficult. Stein Variational Ergodic Search (SVES) was proposed to mitigate this problem, leveraging Stein Variational Gradient Descent (SVGD), which approximates the distributions of trajectories in a particle-based fashion. However, the choice of kernel in SVGD can have a huge impact on the diversity of the trajectories considered. In this paper, the authors propose to use the signature kernel, which can better discriminate between trajectories and respects its Markovian structure. In addition, the authors propose to use Stein Variational Covariance Matrix Adaptation Evolution Strategy (SV-CMA-ES) instead of SVGD, which is gradient-free and better take advantage of GPU parallelism. This is motivated by both the fact that gradients can be expensive to compute and that search-based strategies have shown empirical benefits over gradient-based methods. To estimate the gradient of the signature kernel, they propose to use Simultaneous Perturbation Stochastic Approximation (SPSA), which is also a gradient-free method that evaluates a black-box function via random perturbations. They evaluate SV-CMA-ES across two benchmark systems and five choices of kernels (including the signature kernel). Their results show that trajectories produced via the signature kernel are indeed more diverse, lower cost, and slightly more ergodic than other choices of kernels. They demonstrate improved convergence of SV-CMA-ES over SVGD on both benchmark systems. Finally, they use the signature kernel in SV-MPC, a model predictive control algorithm which leverages SVGD to perform the updates. They show that the signature kernel improves yields trajectories with more diversity while remaining low cost. They also found that the Markov RBF kernel failed to converge to meaningful solutions.", "questions": "- How do the computational costs of each choice of kernel compare? What about the cost for SV-CMA-ES over SVGD? I know this depends a lot on the choice of hardware, but some numbers would help give a ball-park idea.", "limitations": "- The authors provide a number of limitations, mostly centered around computational cost. They also talk about how the repulsive force with the signature kernel can be almost too strong, encouraging poor choices, such a collisions into obstacles. They also provide some remedies for these issues and possible future directions.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- Efficient exploration of a predefined region in a timely manner is an important and core problem in robotics. This paper combines existing tools in a novel way to solve this problem, with clear benefits in terms of trajectory cost and diversity on two simulated platforms. They also show their choice of kernel can be used in an MPC loop, with clear benefits over other choices of kernels.\n- They perform sufficient ablations that show the concrete benefits of each of their design choices. In particular, they evaluate four different choices of baseline kernels compared with their choice of signature kernel. They also explore the convergence of SV-CMA-ES and compare it to SVGD, showing the benefits of this choice of optimizer.\n- Figure 1 is nicely placed and immediately introduces the reader to the empirical benefits of their approach, with a clear improvement in the diversity of trajectories. Figure 2 also makes it clear which kernel works better for this application.\n- The appendix is contains most of the additional information necessary to understand the paper, making it fairly self-contained.\n- In terms of writing, I enjoyed how the main paper focused on the high-level points, relegating the details to the appendix for the interested reader. I also liked how the core questions of the evaluations were outlined in the introduction of Section 7.\n\nWeaknesses:\n- Using only two benchmark systems on similar tasks is a great starting point, but the evaluations could be more extensive. Exploring systems with different choices of dimensionality, types of dynamics, and obstacle types and configurations would strengthen this paper.\n- Figure 3 and 5 make it difficult to clearly see how each method compares. I know there's a table in the appendix, but the visualization of the data could be much clearer. For instance, pairing up by metric rather than kernel could help make the differences distinguishable. Or you could show the distributions with box or violin plots, instead of averages, dividing each metric into a separate sub-panel. While not critical, it would make it much easier to understand the results.\n- In general, there isn't enough discussion of the results or conclusions drawn. In particular, Figures 4 and 6 provide the comparison in convergence of SV-CMA-ES versus SVGD without any discussion or interpretation of the results in the text. There are also a lot of curves in the plot, with substantial overlap, making it hard to interpret. The paper would be stronger if the discussion of these results was elaborated.\n- It would be great to have a little exposition on the benchmark problems used to evaluate the method. While I like relegating a lot of the details to the appendix, the considered problems do need to be introduced more in the main paper.\n- Model predictive control (MPC) is discussed and used in this paper, without really being defined, at least in the main paper. Since it is an important part of the evaluation, I would suggest providing at least a couple sentences describing how it works and adding citations to SV-MPC in the main paper.\n- Similarly, it would be great if the median heuristic would be defined, or at least given a citation. I know it's a fairly standard practice, but for completeness, it would be good to include.", "was_revised": false, "final_justification": "", "rebuttal": "# 1. How do the computational costs of each choice of kernel compare? What about the cost for SV-CMA-ES over SVGD? I know this depends a lot on the choice of hardware, but some numbers would help give a ball-park idea.}\n\nWe thank the reviewer for pointing out the lack of computational benchmarks in our original submission. We agree that providing concrete estimates of computational costs for different kernel choices, as well as comparing the runtime complexity of SV-CMA-ES versus SVGD, would help clarify the practical implications of our method. We provide the following analysis in response. \n\n## Theoretical Computational Cost Comparisons\n### Kernels Computational cost\nWe provided the computational costs for some kernels in Section 5 and Appendix F. \n1. The time complexity of the RBF kernel is \\(O(Td)\\), but in practice can be as low as $O(d)$ on modern hardware with T threads.\n\n2. The time complexity of MRBF is $O\\bigl(|\\mathcal G| \\cdot O(\\mathrm{RBF})\\bigr)$.\n\n3. The signature kernel, the global alignment kernel and the kernelized DTW can be computed in $O(Td)$ with modern parallelized hardware. \n\n### SV-CMA-ES vs SVGD Computational cost\nThe original Stein Variational Gradient Descent (SVGD) paper (reference [10] from our paper) identifies the gradient of the log‑density as its primary computational bottleneck. Comparing the costs of SVGD and SV‑CMA‑ES is therefore nontrivial, since both methods require evaluating gradients of the kernel and of the log‑density at each update. The key difference lies in the cost of obtaining the log‑density gradient itself. When that gradient is easy to compute or available analytically, SVGD will generally be the more efficient choice. By contrast, SV‑CMA‑ES truly shines when the log‑density is nondifferentiable or expensive to evaluate.\n\nIn practice, computing the log‑density gradient typically entails two passes through the density function (a forward and a backward pass). Evolution‐strategy (ES) updates, however, are highly parallelizable on GPUs. The ES updates require multiple parallel forward passes of the log density, and computing the weighted recombination step for each particle can all be batched on modern hardware (with say Cholesky decomposition). Thus, even though SV‑CMA‑ES performs more sampling “work,” its overhead beyond objective evaluations is often quite small. If the log density is non‑differentiable or costly to backpropagate, SV‑CMA‑ES can actually deliver faster wall‑clock convergence by avoiding gradient calls altogether.  Moreover SV-CMA-ES can have faster convergence rates when compared to SVGD with well-tuned hyperparameter(see reference [25] from our paper).\n\n## Experimental Benchmarks\n\nIn the following, we report mean run-times (over 10 independent runs) for SVGD and SV‑CMA‑ES on a common hardware platform, and discuss implementation caveats. \n\n### Experimental Setup\nThe SVGD implementation was fully JIT‑compiled, whereas SV‑CMA‑ES was only partially JIT‑compiled. No optimizations were applied to the signature kernel, so absolute timings may differ under a more optimized implementation. All experiments were run using the following hardware:\n\n1. **CPU**: AMD Ryzen 7 7800X3D (8 cores @ 4.2 GHz)\n2. **Memory**: 32 GB (@ 5600 MT/s)\n3.  **GPU**: NVIDIA Geforce RTX 5060 Ti\n\n\n### Results\n| Experiments               | Multiscale constrained exploration (Section 7.1) | 3D Coverage (Section 7.2) | SV‑MPC (Section 7.3)         |\n|---------------------------|--------------------------------------------------|----------------------------|-----------------------------|\n| No. of iterations         | 3000                                             | 1500                       | 100                         |\n| Problem dimensions | $T=100, \\ d=2$                            |  $T=50, \\ d=3$           | $T=20, \\ d=2$ |\n| N particles               |                                                  |                            |                             |\n|                           |                                                  | **SVGD with RBF kernel**   |                             |\n| 5                         | 2.83 ± 0.08 seconds                              | 1.45 ± 0.17 seconds        | 0.092 ± 0.0034 seconds      |\n| 10                        | 2.82 ± 0.12 seconds                              | 1.45 ± 0.11 seconds        | 0.093 ± 0.0034 seconds      |\n| 15                        | 2.90 ± 0.14 seconds                              | 1.40 ± 0.09 seconds        | 0.095 ± 0.0078 seconds      |\n| 20                        | 3.11 ± 0.17 seconds                              | 1.46 ± 0.10 seconds        | 0.090 ± 0.0043 seconds      |\n| 25                        | 2.94 ± 0.15 seconds                              | 1.44 ± 0.13 seconds        | 0.094 ± 0.0055 seconds      |\n|                           |                                                  | **SVGD with Signature kernel** |                          |\n| 5                         | 13.83 ± 0.84 seconds                             | 6.32 ± 0.47 seconds        | 0.150 ± 0.0030 seconds      |\n| 10                        | 18.33 ± 2.13 seconds                             | 6.30 ± 0.43 seconds        | 0.159 ± 0.0034 seconds      |\n| 15                        | 14.19 ± 0.41 seconds                             | 4.30 ± 1.72 seconds        | 0.157 ± 0.0033 seconds      |\n| 20                        | 14.36 ± 0.47 seconds                             | 6.13 ± 0.62 seconds        | 0.148 ± 0.0027 seconds      |\n| 25                        | 14.86 ± 0.11 seconds                             | 7.17 ± 0.46 seconds        | 0.158 ± 0.0022 seconds      |\n|                           |                                                  | **SV‑CMA‑ES with RBF kernel** |                          |\n| 5                         | 10.13 ± 0.21 seconds                             | 6.01 ± 0.23 seconds        | 0.353 ± 0.0060 seconds      |\n| 10                        | 10.34 ± 0.10 seconds                             | 6.28 ± 0.13 seconds        | 0.359 ± 0.0064 seconds      |\n| 15                        | 11.55 ± 0.62 seconds                             | 6.30 ± 0.85 seconds        | 0.360 ± 0.0049 seconds      |\n| 20                        | 12.08 ± 0.10 seconds                             | 6.90 ± 0.19 seconds        | 0.371 ± 0.0021 seconds      |\n| 25                        | 12.77 ± 0.22 seconds                             | 6.90 ± 0.94 seconds        | 0.378 ± 0.0102 seconds      |"}, {"review_id": "rTBODVjKV4", "reviewer": "Reviewer_eVaC", "summary": "This paper proposes a new method of ergodic search for robots based on the SVES framework, and it aims to improve the diversity of the generated trajectories. The key innovation is to replace the traditional RBF kernel with the Signature Kernel, which improves the kernel's sensitivity to differences in paths. The paper also introduces a PDE formulation of computing the signature kernel that allows parallel computation. The experiments show that with the signature kernel, SVES has better state coverage and convergence results.", "questions": "Typo:\nYou assume $h_2(x) \\leq 0$ in Line 122, but the last term in Eq. 3 is $\\max(0, h_2(x))$, which equals to 0.\n\n\nQuestion:\n1. Do you have evaluation metrics to measure the trajectory smoothness? I am concerned about whether the signature kernel will result in sudden accelerations or sharp turns.\n2. Could you explain in more detail how the signature kernel differs from the RBF kernel when it comes to comparing two similar trajectories?", "limitations": "The authors addressed the limitations and potential negative societal impact of their work.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. The paper contains both qualitative comparisons and quantitative results of trajectories generated using the RBF kernel and the Signature kernel, illustrating that the Signature kernel helps cover the state space. \n2. Using signature kernels for ergodic exploration is an innovative idea, whereas previously signature kernels were applied to trajectory optimization in robotics, as in “Path Signatures for Diversity in Probabilistic Trajectory Optimisation.”\n3. The paper extends the theoretical framework in the prior work Stein Variational Ergodic Search (SVES). \n\n\nWeakness:\n1. The reason why the signature kernel can improve the state coverage is still unclear. In Line 78, the author claims \"The signature kernel naturally discriminates between **subtly different paths**, thus promoting diversity.\" Can you visualize how the signature kernel helps discriminate the local differences between two similar paths, or provide a detailed explanation about why the RBF kernel fails to do so while the signature kernel can?\n2. The experiments only compare SVES with the signature kernel with SVES based on other kernels. However, there have been plenty of kernel-free methods for the robot exploration problem. For example, Burda et al. (2018) “Exploration by Random Network Distillation” introduces **intrinsic rewards** to improve the trajectories' diversity. Also, the **curiosity-driven** approaches in \"Aggressive Quadrotor Flight Using Curiosity-Driven Reinforcement Learning\" and \"Curiosity-Driven Reinforcement Learning based Low-Level Flight Control\" also help improve the state coverage during training. I am confused about whether the SVES framework is necessary for the ergodic search problem.\n3. The experiments are only conducted in the 3D Coverage Drone Experiment, which follows a similar setting in Lee et al. (2024) \"Stein Variational Ergodic Search\". I am concerned about the applicability of the signature kernel in other robotic tasks.", "was_revised": true, "final_justification": "Issues resolved: \n1. Explaining the difference between the RBF and signature kernel in differentiating two trajectories: The authors provided a toy example, and I believe the authors should further include a figure to better visualize it.\n2. Explaining how their method affects the trajectory smoothness: I find that the authors specify the hyperparameters controlling the smoothness in Eq. 42, Appendix J, which is the complete loss function they use. I think the authors should move it to the main text (Section 3), because Section 3 omits part of their loss function.\n \nIssues unresolved:\n1. Compare SVES with the signature kernel with the kernel-free baselines: The authors did not include the recent robotic exploration methods without kernels. Although the authors claimed that they addressed it in the Introduction part, I think the necessity of using the SVES framework in robotic exploration tasks isn't well established yet. Therefore, I think it's better to directly including the kernel-free methods as the baselines, so that it will be more reasonable to replace the traditional RL framework by their SVES framework.\n2. Including experiments in other environments: The authors only use one environment in their experiments. The authors claimed that their main contribution is to \"demonstrate diversification within SVES\", not to propose a new framework because the signature kernel already exists in existing works (See [11][17]][18] in their paper). Although they provided references showing that SVES framework adapts to other environments, I'm still confused whether the experiment within only one environment is enough to \"demonstrate diversification\".\n\nTherefore, I raise my rating to 4, but I think the authors should address the unresolved issues in their revised manuscript.", "rebuttal": "## Note of Gratitude \n\nWe thank the reviewer for the response and the insights provided that can help strengthen the paper.\n\n# 1. Do you have evaluation metrics to measure the trajectory smoothness? I am concerned about whether the signature kernel will result in sudden accelerations or sharp turns.\nThe strong repulsive force provided by the signature kernel may at first seem like a recipe for producing rough and non-smooth trajectories, as producing rough trajectories is one way to boost diversity. However, we found that this did not have an adverse effect on the smoothness. We include a smoothness penalty in the cost/loss function $\\mathcal J$ to enforce trajectory regularity (see Sections 2 \\& 3 and Appendix J and the resulting trajectories are visualized in Figure 1, Section 7 and Appendix I) and SVES guarantees convergence to a locally optimal trajectory with mild conditions on the chosen kernel (that signature kernel satisfies). The proof of convergence is provided in Appendix A of [1]. So, the trajectories should converge to a set of smooth trajectories provided that there is a smoothness penalty in the cost/loss function.\n\nAfter the rebuttal, an ablation study that tunes the trajectory smoothness and compares the performance of the kernels with respect to the trajectory smoothness will be provided along with the proof of convergence.\n\n# 2. Could you explain in more detail how the signature kernel differs from the RBF kernel when it comes to comparing two similar trajectories?\n\nWe provide a brief mathematical reasoning for why the signature kernel makes a good kernel, it is discussed in lines 154-158 \\& 768-775 (the reviewer may also refer to references from [11,14,15,19,61,62] from our paper). A short and oversimplified answer is that the signature transform is injective  on all non-tree-like paths, meaning no two such paths share the same signature. But this is not the case for the RBF kernel. Consider paths:\n1. $p_1(t) = (x_1(t),y_1(t)) = (t, 0) \\quad t \\in [0, 2\\pi]$\n2. $p_2(t) = (x_2(t),y_2(t)) = (t, sin(t)) \\quad t \\in [0, 2\\pi]$\n3. $p_3(t) = (x_3(t),y_3(t)) = (t, sin(t + \\pi)) \\quad t \\in [0, 2\\pi]$\n\n\nIt is easy to see that $k^{RBF}(p_1,p_2) = k^{RBF}(p_1,p_3)$. But this not the case with the signature kernel due the the signature transform being invective up to “tree‑like” equivalence.\n\n# Weakness\n## Weakness 2: \nThis is briefly discussed in Section 1 of our paper. In short, classical ergodic planners focus on steering a single trajectory (or a centrally controlled fleet) to match\n    a target spatial distribution. In contrast, our SVES‐diversification methods generate an set of distinct or diverse ergodic trajectories in parallel, leveraging repulsive interactions or signature‐based seeding to ensure both high coverage and trajectory diversity in a unified optimization. Refer to Section 1 \\& 2 of [1] and Section 1 and Appendix A of our paper for a more formal take on this. The reviewer can also choose to refer the response provided to reviewer KSg3.\n## Weakness 3: \nBecause our primary goal is to demonstrate diversification within SVES, we intentionally replicated the ergodic search experiments from the original Stein Variational Ergodic Search paper. Throughout the paper, we also also include references to prior successes of the signature kernel in robotics applications, for example:\n1. reference [11] from our paper used the signature kernel to produce a diverse set of solutions in motion planning tasks such as manipulation.\n2. reference [17] from our paper used the signature transform for Human Action Recognition.\n3. reference [18] from our paper used the signature kernel for trajectory following/stabilization.\n\nIn addition, the reviewer may also refer to [2], where the authors used the signature transform in data driven control. \n\n# Typo:\n## Clarification of the Constrained vs. Penalized Formulation\n\nIn the main text we first posed the constrained optimization problem as\n$$ {\\text{minimize}} \\quad \\mathcal{E}_\\pi(x(t)), \\qquad \\text{subject to} \\quad h_1(x) = 0, \\quad h_2(x) \\le 0, \\quad \\forall t \\in [0, T].  \\qquad \\quad (1)$$\n\n\nThere is no typo in Eq 3 of our paper; instead, we switch to an unconstrained,\n*penalty‑augmented* version of (1) by adding a hinge‐loss\nterm that *penalizes* any violation of the inequality constraint.  Concretely,\nwe define\n\n$$\\mathcal J_\\pi(x)\n        =\\mathcal E_\\pi(x)\n        +\\rho(x)\n        +c_1\\,h_1(x)^2\n        +c_2\\max (0,h_2(x)). \\qquad \\quad (2)$$\n\n## Why this is correct and not a typo:\n1. The hinge function $\\max(0,h_{2}(x))$ is $0$ whenever\n        $h_{2}(x)\\le0$, i.e. the original constraint is satisfied.\n2. If $h_{2}(x)$ becomes positive (constraint violation),\n        $\\max(0,h_{2}(x))=h_{2}(x)>0$, so the penalty term\n        $c_{2}\\,h_{2}(x)$ grows linearly, discouraging infeasible\n        solutions.\n3. Thus (2) is a standard “soft‐constraint” or\n        “penalty” reformulation of the hard constraint in\n        (1), not a mistake in the sign or definition.\n\n# References \n\n[1] Darrick Lee, Cameron Lerch, Fabio Ramos, and Ian Abraham. Stein Variational Ergodic Search. arXiv\npreprint arXiv:2406.11767, 2024. https://arxiv.org/abs/2406.11767\n\n[2] Anna Scampicchio and Melanie N. Zeilinger. On the role of the signature transform in nonlinear systems\nand data-driven control. arXiv preprint arXiv:2409.05685, 2025. https://arxiv.org/abs/2409.05685"}]

Author rebuttal:

Meta-review: The paper presents ergodic search for robots based on the Signature Kernel Evolution Strategy (SVES) to produce more diverse trajectories.  So they modify an existing methods by plugging in a different kernel (replacing an RBF kernel) and a PDE formulation that allows parallel computation.  The reviewers suggest to improve the paper by adding more evaluation, e.g., not only comparing against other kernel-based methods, but also against kernel-free methods.  Furthermore, evaluation on more environments might be useful.  Nonetheless the paper is well written and the overall opinion of the reviewers is borderline accept, so I suggest an "accept".
Decision: Accept (poster)