Memory-Augmented Potential Field Theory: A Framework for Adaptive Control in Non-Convex Domains
Authors: Dongzhe Zheng
Wenjie Mei
Venue: NeurIPS 2025 poster ()
Keywords: Stochastic optimal control
non-convex landscapes
potential field
model predictive path integral
Primary area: optimization

Abstract: Stochastic optimal control methods often struggle in complex non-convex landscapes, frequently becoming trapped in local optima due to their inability to learn from historical trajectory data. This paper introduces Memory-Augmented Potential Field Theory, a unified mathematical framework that integrates historical experience into stochastic optimal control. Our approach dynamically constructs memory-based potential fields that identify and encode key topological features of the state space, enabling controllers to automatically learn from past experiences and adapt their optimization strategy. We provide a theoretical analysis showing that memory-augmented potential fields possess non-convex escape properties, asymptotic convergence characteristics, and computational efficiency. We implement this theoretical framework in a Memory-Augmented Model Predictive Path Integral (MPPI) controller that demonstrates significantly improved performance in challenging non-convex environments. The framework represents a generalizable approach to experience-based learning within control systems (especially robotic dynamics), enhancing their ability to navigate complex state spaces without requiring specialized domain knowledge or extensive offline training.

Memory-Augmented Potential Field Theory: A Framework for Adaptive Control in Non-Convex Domains
Dongzhe Zheng
Wenjie Mei
Abstract
Stochastic optimal control methods often struggle in complex non-convex land- scapes, frequently becoming trapped in local optima due to their inability to learn from historical trajectory data. This paper introduces Memory-Augmentee tential Field Theory, a unified mathematical framework that integrates historical experience into stochastic optimal control. Our approach dynamically constructs memory- based potential fields that identify and encode key topological features the state space, enabling controllers automatically learn from past experiences and adapt their optimization strategy. We provide a theoretical analysis showing that memory -augeeed potential fields possess non-convex escape properties, asymptotic convergence characteristics, and computational efficiency. We imple- ment this theoretical framework Memory-Augmentted Model Predictive Path Integral (MPPI) controller that demonstrates significantly improved performance in challenging non-convex environments. The framework represents a general- izable approach to experience-basee learning within control systems (especially robotic dynamics), enhancing their ability navigate complex state spaces without requiring specialized domain knowledge or extensive offline training.
Po-
Introduction
Stochastic optimal control has proven highly effective for handling nonlinear systems and uncertain environments, finding widespread application in robotics, reinforcement learning, and complex system control. Among these approaches, Model Predictive Path Integral (MPPI) control stands for its ability to handle continuous state-action spaces through stochastic sampling and exponentially weighted averaging. However, these methods still face significant thcoretical and practical challenges when confronting highly non-convex value function landscapes.
From optimization perspective, stochastic optimal control problems be viewed trajectory optimization over a value function landscape. When this landscape exhibits complex non-convex characteristics, optimization processes may become trapped in local optima, unable reach global solutions. While introducing noise sampling (as MPPI's random perturbations) can somewhat mitigate this issue, significantly non-convex features often lead to inefficient sampling or control instability when noise is simply increased.
From dynamical systems perspective, non-convex value functions correspond to systems with multiple attractors and unstable equilibrium points. Control algorithms need to identify these features and, when necessary, guide the system across energy barriers to escape suboptimal attractor regions.
Dongzhe Zheng is with the Department of Computer Science and Engincering, School of Electronic Information and Electrical Engineering. Shanghai Jiao Tong University, Shanghai 200240, China. 1 Wenjie Mei with the School Robotics and Automation, Suzhou Campus, Nanjing University, 1520 Taihu Avenue, Suzhou 2155663, China. Correspondence to Wenjie Mei (F-mail: mei. wenjie@njuu edu.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Traditional stochastic control methods have limited capabilities in this regard, as they lack awareness and memory of the state space's 's topological structure.
Traditional stochastic optimal controllers lack learning from past trajectories. This design means controllers might repeatedly fall into the same suboptimal regions, failing to extract experience from previous "failures." In contrast, advanced cognitive systems (like humans) dynamically adjust decision strategies based on prior experience when exploring complex environments.
solely on current states without
This paper addresses fundamental question: How can we integrate "memory" mechanisms into stochastic optimal control frameworks, enabling controllers to automatically learn state space topological features from historical trajectories and adjust optimization strategies accordingly? We introduce Memory-Augmentt
Potential Field Theory, integrating historical state experience into stochastic optimal control through dynamic potential fields that automatically identify and encode topological features of the state space during execution. These fields act as correction terms to reshape the value function landscape, enabling adaptive navigation of non-convex optimization problems. We provide theoretical analysis showing that, under standard assumptions,
potential fields admit (i) probbbbbbbbility escape from local minima, (ii) asymptotic convergence guarantees, and (iii) low additional computational overhead. Our framework provides: 1) automatic detection and encoding of problematic regions like local minima and low- graddardie areas, 2) dynamic reshaping of value functions for efficient escape from suboptimal attractors, 3) convergence to neighborhood of the global optimum with high probability under stated assumptions, and 4) significant performance improvements in complex control tasks without requiring extensive offline training.
Our approach uniquely integrates memory mechanisms with dynamical systems theory and stochastic optimal control, analyzing memory's 's impact on non-convex optimization topologically. Beyond sim- ply storing experiences, our method automatically identifies key state space features and dynamically reshapes value function landscapes, enabling "meta-optimizatioon capabilities under fixed-budget online control settings where each method receives the same number of environment interactions. The code has been anonymized and is available at sattps://gitann com/ Cont www.einnnnnnrrrnneo /MAPTT. MPPI
Merrary Modile
Mih MPPF)
Pedfuee Univetoe
Faten Fold
of
Figure 1: MA-MPPI framework flowchart showing the integration of memory modules with standard MPPI. The pipeline augments stochastic control with experience- based potential fields that enable navigation through complex non-convex environments and escape from local minima.
2 Related Work
Stochastic optimal control and path integral methods form our foundation. Path Integral Control approximates Jumilton-Jacobi-Bellle developed Model Predictive Path Integral (MPPI) control, combining path integral techniques with model predictive control. Theodorou et al. [19 analyzed this approach from an information geometry perspective, connecting to relative entropy optimization. While effective for handling nonlinearities. these methods struggle with highly non-convex value functions. Recent MPPI variants that address non-convexity include LOG-MPPI [33 and DRPA-MPP 4 which improve exploration or add reactive repulsion without persistent memory; our approach is complementary by learning persistent topological features over time. Covariance and temperature design for sampling- MPC/MPPI
equations through Monte Carlo sampling. Williams et al. [21]
has also been studied [23 22 our temperature modulation induces equivalent covariance scaling
2

---

within the path integral weighting. Extensions to constrained and smooth variants trajectory quality rather than topological learning.
[ 10 focus on
Non-convex optimization approaches include simulated annealing, stochastic gradient Langevin dynamics, and entropy regularization. Zhang et al. [24] studied energy landscapes and critical paths in non-convex problems. Jin et al. [9] proved noisy gradient methods can escape strict saddle points in polynomial time. These foundations rarely incorporate learning from historical trajectories to improve subsequent optimization.
Dynamical systems and potential field methods frame control as designing vector fields guiding system states toward convergence. Koditschek and Rimon [[]] pioneered navigation function topological properties, proving conditions for globally asymptotically stable control laws. Traditional potential fields, while theoretically elegant, typically rely on fixed potential forms lacking adaptivity, unlike experience- based approach.
Memory-augmenter learning has expanded in reinforcement learning through Experience Replay for improving sample efficiency. Pritzel et al. [15] proposed Neural Episodic Control, accelerating learning by remembering previously visited states. In control theory, Heess et al. 77 explored memory- augmented controllers for partially observable environments, but few works analyze memory's impact from dynamical systems perspectives.
Memory-Augmented Potential Field Theory
3.1 Formulation of Stochastic Optimal Control
Consider a discrete-time stochastic dynamical system:
1 = f u+) + ct.
4 ~~~000 ~(,
(1)
where T, E RR is the system state, u, E I is the control input, and E gaxx is the noise covariance matrix. The objective is to find a control sequence u {uo, U1,... WT that minimizes the expected cumulative cost:
1 J(n) Σ c(a,, ue) + cr(
where R R R is the immediate cost and cr R" + E is the terminal cost. Through the path integral control framework, the optimal control can be expressed as:
" uc(T)p(F(zz)ddr
where T represents trajectory sequence starting from Fı, and p(r (2:) is the trajectory probability distribution:
p(T(a.)) = 7(xz) exp (-)
with S(T) representing the total trajectory cost, A > 0 controlling exploration-exploitttio tradcoff, and Z(x,) = JT exp (-)) dr as the normalization constant.
3.2 Memory-Augmented Potential Field Framework
Our framework extends the standard value function with memory-dependent term:
V(r, M) = a(x, M) Vause (x) + (1 a(x, M)) Vmmm (æ, M)
(5)
3

---

where M represents memory of topological features,
Fundden
V Vasse I R is the original task objective, Vmen' R x M + IR incorporates historical information, and a: IR" x M [0, balances these components based on proxim-
ity to memorized features.
The memory M consists of elements representing topo- logical features:
Locat
Venory-Augreraa Volue Punction
M = {(m, Ti Yo, Ni, d:) li -i= .,., [M]} (6)
where m, E R" is the feature position, 7, R$ is the influence radius, % R$ is the feature strength, R2 E (1,2,3) identifies feature type (local minima, low- gradient region, high-curvature region), and d; E R" provides a direction vector for applicable features. The memory evolves during execution through update function Lt: M = (M;, Vz, Ex)
Valeo tongree Hy prott Nemdure selern Orijicl Sciwe lafor gioit) Menoyy Poter voluce orggaal
veleend
Onlecial Monnen
Figure 2: Original vs. augmented value functions.
memory-
(7)
where &. contains the topological features extracted from state and context.
The memory potential field is constructed as:
[M
Vmmem (1, .M = Σ Ø(I, m. Te, K4, d:)
(8)
2=
where é is basis potential function tailored to each feature type. Detailed potential construction methods are in Appendix B.3
As Figure 2 shows, our approach transforms the original value function into a memory--aummented one and creates smooth optimization paths (blue trajectory).
3.3 Theoretical Properties
We establish several key theoretical properties that guarantee the effectiveness of nemory-augmente potential fields in non-convex control problems.
Theorem 3.1 (Non-convex Escape Property). Let B(m,,r;) fa R" || mill r; be local minimum region recorded in memory M. If > 17 Sup=eB(mm.y) (Vaaae())) for some constant 0, then for any confidence level 0 < 8 < 1, there exists finite time T Tescape (6) < oo such that
P(t < Tessape(( (6): :z & r.) xo B(m,, r.) 21
This theorem guarantees that with sufficiently strong memory features, the system escape local minima finite time with high probability. The memory potential creates an "outward push" that overcomes the "inward pull" of the base value function.
Theorem 3.2 (Asymptotic Convergence Property). Let x² = arg minzeR" Vhose (x) be the global optimum of the base value function. Assume Vbuse is coercive and satisfies: limms Vouse (x) For any 0 and confidence level 0 < 8 < 1, there exists a finite time T Teom (e, 8) < oo such that
P inf (e.Tear(e.))
||xe- x=11 S
2 118
(10)
Despite altering the value function landscape, memory augmentation preserves convergence to the global optimum because memory effects primarily impact identified problematic regions while maintaining the original behavior elsewhere.
Theorem 3.3 (Adaptive Learning Efficiency). Let t= {L1, L2, ..., LK} denote K independent local minimum regions in the state space. Let MPPA and TMPPI be the expected times for MA-MPPI and standard MPPI to reach the global optimum. Then:
TmPPI 2 ((K).
(11)
where D( ()K denotes a lower bound that grows at least linearly with K.

---

This theorem shows that memory augmentation efficiency increases with the number of local minima, as our method avoids revisiting known problematic regions while standard approaches repeatedly encounter the same traps. Our approach connects to Morse theory [1] by dynamically modifying value function Morse indices, transforming local minima into saddle points while preserving global optimum attraction. Complete proofs for the above theorems are provided in Appendix C
4 An Extension: Memory-Augmented Predictive Path Integral Method
The MPPI control, as a sampling-based MPC variant, evolves into Memory-Augmented MPPI controller, a practical implementation addressing traditional MPPI's susceptibility to local optima in non-convex environments. This section details MA-MPPI's design, components, and workflow, showcasing the theory's 's transformation into effective control technology.
4.1 System Architecture and Algorithm
MA-MPPI comprises four functional modules: 1) an MPPI control core implementing sampling- based optimization, 2) a topological feature detector identifying critical features from trajectory data, 3) memory representation that stores and evolves spatial information, and 4) an adaptive potential field synthesizer that modifies the value function based on memory.
Poternaal change During Novigatinn
The workflow begins with state observation followed by feature detection through trajec- tory analysis. The controller then updates its memory representation M,, storing location, radius, strength, and type information for each significant feature according the update rule
myy feteria
120
vepo
130
M1+1 = [[M,, Zt, Et).
Dentrol Wesght Adaptation
As shown in Figure 3] the system automatically transitions between normal navigation and es- cape behavior when detecting stuck situations. The top plot illustrates how different poten- tial components (goal, memory, trap) combine into the total potential, while the bottom plot demonstrates the adaptation of control weights during execution. During stagnation periods (highlighted in pink), goal weight decreases while memory and trap weights increase, en- abling escape from local minima without ex- ternal intervention.
334
Megil
Figure 3: Potential components and control weights during MA-MPPI execution, showing automatic phase transitions in response to detected stagnation (pink regions).
Using the updated memory, MA-MPPI synthesizes an enhanced value function:
Va,, M,) = (2r, Mosso Vasso()) (1) + (1 a(x, M,)) Vieem (ar, M,)
(12)
where (xx M,) balances the influence between base objective and memory potentials. The controller also adjusts the sampling temperature:
X(r,, M,) = Ao + 7 axx, Mc)))
(13)
This temperature increase equivalent to a proportional inflation of sampling covariance E,,, yielding broader perturbations in memorized regions.
The algorithm then executes standard MPPI: generating control signals, simulating trajectories, evaluating costs, and determining the optimal control through weighted averaging:
K
exp(- S(++)
(14)
2 exp(- X S(T)))
The computational complexity remains O(K Н n + [M])) with memory operations typically representing minimal overhead as M < K H n. Detailed algorithmic implementations are provided in Appendix E
5

---

4.2 Topological Feature Detection
Topological feature detection allows MA-MPPI identify and remember critical structures that im- pact optimization. The system recognizes three feature types that correspond to different optimization challenges: local minima where controllers become trapped, low-gradient regions where progress slows, and high- high-curvature regions requiring precise navigation.
Detection employs trio of complementary mech- anisms: state stagnation analysis, gradient exam- ination, and curvature assessment, which collec- tively map challenging regions in the optimization landscape, while continuously refining detected fea- tures through balanced process of incorporation (when encountering problematic regions), consoli- dation (merging similar features for representational compactness), and dynamic importance adjustment (based on encounter frequency). This comprehen- sive topological mapping enables the controller to anticipate obstacles that typically confound tradi- tional approaches, while ensuring computational efficiency and focusing memory resources on per- sistently challenging areas.
Reolopical "eatur Datection
In practice, thresholds are initialized by scaling with state statisties (0var 0.01 Var(zr), Ograd the 10- 20th percentile of [ observed in warm-up rollouts, eerrv via Hessian aaditiin-numbee per- centile), and then tuned within +25% without ma- terial performance change (see Sec. J).
Figure 4: Topological feature map showing de- tected navigational challenges.
In Figure 4 contour lines represent the potential landscape, blue circles indicate obstacles, red outlines show local minima, and arrows depict the gradient field. This figure illustrates the system's environmental mapping capabilities, identifying regions where traditional navigation would fail. By recognizing convergent gradient patterns forming "valleys" and narrow passageways in the potential contours, the controller builds an increasingly accurate model of the environment's challenging characteristics with continued operation.
This topological knowledge enables MA-MPPI to anticipate difficulties before encountering them, adaptively modifying both the value function landscape and sampling strategy navigate complex environments more effectively, The technical details of detection mechanisms, feature classification, consolidation algorithms, and dynamic memory management are provided in Appendix F
4.3
Adaptive Potential Field Synthesis
The enhanced value function combines the base objective with memory-derived potentials through an adaptive weighting mechanism:
(a, M,) = axx Mt) (x) + (1 a(x; Mc)) Mc)
(15)
where axx M,) E [0, 1] balances the influence between objectives based on proximity to memory features.
The memory potential Vmeem employs type-specifir implementations for different topological features: radially decreasing functions for local minima, directional guidance functions for low-gradient regions, and saddle-poin functions for high-curvature areas. This type-specific approach enables tailored responses to different environmental challenges.
Beyond value function enhancement, the system also dynamically adjusts the MPPI temperature parameter via
X(x, M,) Ao (1 + 77 (1 a(x, M.))
(16)
This dual-layer adaptation increases exploratory behavior near problematic regions, enhancing the system's 's ability to escape local optima through intelligently directed sampling. Technical details

---

The adaptive potential field synthesis mod- ule transforms memory into control influence, bridging the memory system with the control algorithm. This critical component generates enhanced value functions that enable efficient navigation around known problematic areas while maintaining global objective pursuit. As shown in Figure 5 the memory module generates a spatially-aware force field that guides the robot away from previously prob- lematic regions. The orange and yellow ar- rows represent repulsive forces, with color intensity indicating magnitude. Red circles mark identified trap regions with their asso- ciated strength values, showing how the sys- tem's 's experience shapes its navigational be- havior.
thate
Peatton
Figure 5: Memory-based potential field visualization showing repulsive forces (arrows) and identified trap regions (purple circles).
regarding potential function formulations, feature-specific implementations, and computational optimizations are provided in Appendix
5
Experimental Evaluation in Robotic Control Environments
We evaluated MA-MPPI on benchmark robotic control tasks, comparing against state-of-the-an algorithms to validate the advantages of memory augmentation in complex control landscapes.
5.1 Experimental Setup
We selected four environments of increasing complexity: Pendulum-vl (2]. BipedalWalker-v3 (2]. HalfChectah-v4 (20). and Humanoid-v v4 [20]. These environments present varying degrees of non- convexity and dimensionality, ranging from the simple pendulum swing-up to 376-dimensional humanoid control task with numerous local optima.
MA-MPPI was implemented with environment eivironmenttapprooriiat appropriate prediction horizons: 15 steps for Pendulum-vl, 20 steps for BipedalWalker-v3. 25 steps for HalfCheetah-v4,, and 35 steps for Humanoid-v4, reflecting the increasing dynamics complexity. We compared against standard MPPI 21. modern reinforcement learning approaches (SAC [S) PPO [16), DDPG [8), and traditional op- timal control methods (iLQR [18), MPC [14). All methods operate under the same
budget of 2000 environment steps. For model-free RL /SAC/PPO/DDPG), we do not allow extra environment interactions beyond this budget; training is strictly on on-policy/within-be
to ensure fairness to sampling- based controllers that already perform heavy internal simulations per step. For all experiments, conducted 30 independent with different random seeds, each consisting of 2000 control steps. Detailed environment specifications and implementation settings are provided in Appendix H.I
Our evaluation protocol consisted of two phases: an Adaptation Phase measuring learning efficiency during the first 500 environment interactions, and Stability Phase assessing asymptotic performance after 2000 total interactions. This approach enables fair comparison between methods with different learning characteristics.
5.2 Results and Analysis
We report learning curves over the 2000-step budget, and summarize performance via (i) AUC-2000 (area under the learning curve over 2000 steps), and (ii) Final-2000 (average return over the last 200 steps). This avoids extrapolating asymptotes inappropriate for within-budger RL.
7

---

Table I: Performance comparison: Average cumulative rewards (#
Table I presents the asymptotic performance comparison. MA-MPPI demonstrates consistent improvements under the fixed-budget setting across all environments, with the advantage amplifying in more complex domains. The performance gap is particularly pronounced in Humanoid where MA-MPPI outperforms the best RL method (i.e., SAC) by 27%.
represents standard deviation).
Method
Pendulum-vi
BipedalWaller-v3
HaliChestab-v4 v4
Humamoid-v4
MA-MPPI (Ours)
-152.4=9.7
298.4:77.2
5893.7:156.4
4978.5:
MPPI
1165.8110.99
241.7419.1
1427.91148.6
2914.24318.7
SAC
1192.6=:113
112.6:22444
1763.4+224.2
936.74:354.9
PPO
.205.14:1..0
96.3431.77
1287.51.237.9
612.31329.66
-214.7+13.5
148+335.9
1149.24261.8
381.4--402.7
iQQR
-258,91155
184.24226.5
3614.82:205.7
1927 St:411.2
MPC
1188.3+10.6
219.6+222.3
4127 1922.6+192.3
2784. 14:306.8
Key robustness results (lo- cal minima escape) are
Table 2: Local optima escape rates (%) from challenging initial states.
shown in Table 2 (main
Method
Pendulum-vi
Hipeda/Wallee--3
HaltChestab-vd
Humamoid-v4
MA-MPPI (Ours)
89.2.4.11
83.315.2
76.8164
72.31788
text). We define Local Op-
MPPI
48.3+5.77
41.62644
36.24722
29.4886
tima Escape Rate (spppa
65.7+448
58 88.345.7
46.747.4
PP()
59.4+55.2
45.3+7.3
as the percentage of suc-
DDPG
53.8:1..6
46.926.55
39.7477
32.5+8.7
cessful escapes from prede- fined trap states, where a
ILOR
27.6.16.3
21.417.2
16.318.1
11.8193
MPC
42.545.9
38.316.7
trap state is defined any state from which the expected return falls below 50% of the maxi- mum achievable value, and the agent remains within a small neighborhood for at least 50 time steps without improvement. As shown in Table MA-MPPI achieves significantly higher escape rates across all environments, demonstrating that memory augmentation effectively reshapes the value landscape around trap states, creating "tunnels" that guide the controller toward more promising regions.
During normal operation, standard MPPI encountered trap states approximately 2.8x more frequently than MA-MPPI (5.7 vs. trapped episodes per 100 episodes), confirming the proactive trap-avoidance capability conferred by spatial memory. For detailed trap frequency analysis and occurrence patterns across environments, see Appendix H.3
Ablation studies reveal memory as the critical component (42-58% performance decrease when removed), with increasing importance complex environments (see Appendix HH.4 for complete component analysis). Our hyperparameter sensitivity analysis demonstrates the algorithm's s robustness to parameter variations, with performance generally stable within +25 parameter ranges (see Appendix[]] MA-MPPI's advantages stem from trap identification and avoidance, value landscape reshaping, and memory-guides exploration, all with modest computational overhead (12-18%, detailed in computational performance analysis in Appendix H.5))
Control quality analysis revealed an unexpected benefit: the production of smoother control trajectories with fewer oscillations, yielding more mefry-efficieent motion, particularly valuable for physical robots. The online adaptation capability provides fundamental advantages for deployment in unknown environments, contrasting with RI. methods that require extensive offline training (see comparative learning analysis in Appendix). This addresses a fundamental limitation in sampling- based control: the inability to learn from past failures.
The memory term only reshapes the objective; it composes with constrained/smoothed MPPI. In our pilot study, MA+Constrained-MPP improved repetitive-task success by ~55% at unchanged violation rate; MA+Smooth-MPPH cut escape time by ~33 while preserving smoothness.
Further Experiments on Complex Engineering Systems
To validate the effectiveness of our
Potential Field Theory in worl- domains, we conducted two types of experiments on complex systems: 1) a power system control problem and 2) unmanned aerial vehicle (UAV) obstacle avoidance task.
6.1 Evaluation Methods
We compared our MA-MPPI approach against several state-of-the-at methods: standard MPPI 21 Diffusion Policy B3, Motion Transformer [17], MLP-based MPC [14), and DKO-based MPC [2 6
8

---

Performance evaluation focused on success rates, solution optimality, computational efficiency, and local minima escape capability. See Appendix I.T for detailed experimental protocols.
Cost functions are harmonized across controllers:
U'goal -lpp-llp---- 1lppppp-p p-- + Wobs" Σ ((dmimma Obstacles))
E 1 W
Σ llue up-1 11
(17)
where ((d) [d d rsfee) (T'afe - d)- terminal collision penalty 1000. MPPI-style methods use J directly; RL baselines use reward -J (same weights); constrained MPC uses barrier equivalents; iLQR uses quadratic approximations. Weights were selected by grid search on standard MPPI and fixed across all methods; sensitivity +25 shows change in ranking.
6.2 Experiment I: Power System Control
We evaluated our approach on power system stability using the well-known IEEE 39-bus New England test system, employing the dynamic simulation framework open-sourced in [25]. The power system dynamics are represented as 1 = f far u, d) and y = g(x), where æ E R" is the system state, E I. represents control inputs, d E IRP represents disturbances, and y E IR9 represents measured outputs. The control objective is min E c( u4) + er (#r) subject system constraints.
We tested against three critical disturbances: (1) three-phase short circuit fault on a 345kV transmis- sion line, (2) sudden load increase of 25% at key buses, and (3) trip of 650MW generator. These represent challenging operational scenarios requiring rapid response while maintaining stability.
We assessed performance using constraint vio- lation rate V. N₂. 100%, economic effi- ciency E, = Coptsmal stability margin Sm min; dist(æ;, OS), computation time Te, and dis- turbance recovery time T,. See Appendix [.4 for detailed experimental parameters.
Cactual
F shows system performance during three major events: a three-phase fault 04:00, a 25% load change at 10:00, and a 650MW generator trip 16:00. MA-MPPI (blue line) demonstrates superior recovery speed and stability. Shaded areas represent confidence intervals.
Figure 6: Power system stability response to critical disturbances.
Table 3 shows that MA-
Table 3: Performance comparison on power system control tasks
MPPI significantly reduced constraint violations (2.3% vs.
Method
Constraint
Economic Efficlency
Stability Margin
Compute (ms)
Disturbunce Recovery
(%)
5.7% for standard MPPI) while improving economic efficiency (1.06 vs. 1.13) and stability mar- gin (0.18) vs. 0.112). Most importantly, MA-MPPI achieved faster disturbance recovery (4.2s vs. 8.7s), particularly during severe events like generator trips.
MA-MPPI (Ours)
2,3
1,06
0.187
34,5
4.2
Standard MPPI 21
5,7
1.13
0.112
27.8
8.7
Policy
Motion Tranformer
3.5
1.08
0.158
52.1
5.5
LILP-based MPC
14
6.2
1.16
0.103
31.2
9.6
6.3 Experiment UAV Obstacle Avoidance
The environment featured cylindrical obstacles cre- ating navigation scenarios with narrow passages and potential local minima. As shown in Table 4 MA-
Table 4: Performance comparison on UAV obstacle avoidance (aver- aged over 100 trials)
Method
Success Rate (%)
Puth Optimality
Control Smoothness
Compute Time (ms)
Local Minima
Escapes (%)
MA-MMPLL/Oum
Standard MPPI 21
72.8
1.45
0.34
10.2
34,2
Diffusion Policy
B
79.6
1.37
0,31
18.4
56.8
MPPI significantly outper- formed baseline methods, with 94.3% success rate
Transformer
0.29
MLP-based MPC
4
68.2
1.53
0.42
14.3
41,9
DRO-based MPC
PP]
76.4
1.31
0.38
15.8
49.3

---

We implemented a physics-based UAV simula- tion with realistic aerodynamies following the standard quadrotor model:
Legene
1
- R. -kaalllllo m
R.
(18)
W J (T x Jw)
where R is position, R E SO(3) is orientation, w E IR is angular velocity, f E R is thrust, mass, inertia matrix, y is gravity, ka is drag coefficient, and w is the skew-symmetrie matrix of w.
IR3 is velocity,
UAV obstacle avoidance with MA- MPPI. The visualization shows the reference path (blue), valid trajectory rollouts (green), col- lision rollouts (red), the selected optimal trajec- tory (yellow), and target waypoints (magenta),
Figure 7:
R is torque, is
compared 72.8% for standard MPPI. Most notably, MA-MPPI achieved an 87.5% local minima escape rate, far exceeding standard MPPI's 34.2%. This translates to better path optimality and control smoothness with only modest computational overhead. For detailed performance analysis, refer to Appendix
7 Discussion and Limitations
Although Memory-Augmente Potential Field Theory demonstrates robust performance across domains through its topological learning and adaptive optimization capabilities, several limitations remain. The current MA-MPPI approach shows restricted generalization between similar features (see Appendix K), lacks sophisticated memory management for extended operations, and doesn't leverage multi-agent knowledge sharing.
(i) We assume full-state feedback; extending to state-estimation uncertainty (e-g., UAV localization noise) via memory-aware filters future work. (ii) Rapidly time-varying traps require faster decay and change-poine detection; our dynamic environment pilot supports this but full theory remains open. (iii) semantics-induced traps motivate learning- based feature detectors; we plan to hybridize rule-based topology with learned representations. Importantly, Ceemory-Augmeniee Potential Field Theory is not intended to replace learning- based methods but can complement them: potential integration with reinforcement learning could combine the theory's 's advantageous memory structures with RL's policy optimization capabilities. This hybrid approach could leverage the strengths of both paradigms while addressing their individual limitations in complex non-convex control problems.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant 62403125, in part by the Natural Science Foundation of Jiangsu Province under Grant BK20241283, and in part by the Fundamental Research Funds for the Central Universities under Grant 2242024k30037 and Grant 2242024430038.
10

---

References
[1] Isin M Balci, Efstathios Bakolas, Bogdan Vlahov, and Evangelos A Thcodorou. Constrained covariance steering based tube-MPPI. In 2022 American Control Conference (ACC), pages 4197-4202. IEEE, 2022.
[2] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. OpenAl Gym. arXiv preprint (tfi::160601154444 2016.
[3] Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousincau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, (1:0--11)1184-117000
2025.
[4] Takahiro Fuke, Masafumi Endo, Kohei Honda, and Genya Ishigami. DRPA-MPPI: Dynamic repulsive potential augmented MPPI for reactive navigation in unstructured environments. In 2025 IEEE 2 International Conference on Automation Science and Engineering (CASE), pages 961-968. IEEE, 2025.
[5] Tuomas Haamojj, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning, pages 1861-1870. PMLR, 2018.
[6] Yigiang Han, Wenjian Hao, and Umesh Vaidya. Deep learning of Koopman representation for control. In 2020 59th IEEE Conference on Decision and Control (CDC), pages 1890-1895, JEEE, 2020.
[7] Nicolas Heess, Jonathan J Hunt, Timothy P Lillicrap, and David Silver. Memory-based control with recurrent neural networks. arXiv preprint 1rx1v:1512.04455, 2015.
[8] Nicolas Heess, Gregory Wayne. David Silver. Timothy Lillicrap. Tom Erez, and Yuval Tassa. Learning continuous control policies by stochastic value gradients. In Advances in Neural Information Processing Systems, volume 28, 2015.
[9] Chi Jin, Rong Ge, Pranceth Netrapalli, Sham M Kakade, and Michael Jordan. How to escape saddle points efficiently. In International Conference Machine Learning, pages 1724-1732. PMLR, 2017.
[10] Tackyung Kim, Gyuhyun Park, Kiho Kwak, Jihwan Bac, and Wonsuk Lee. Smooth model predictive path integral control without smoothing. IEEE Robotics and Automation Letters, 10406 10413, 2022.
[11] Daniel E Koditschek and Elon Rimon. Robot navigation functions on manifolds with boundary. Advances in Applied Mathematics, 11(4):412 442, 1990.
[12] Bethany Lusch, Nathan Kutz, and Steven L Brunton. Deep learning for universal linear embeddings of nonlinear dynamics. Nature Communications, 9(1):4950, 2018
[13] Ihab S Mohamed, Kai Yin, and Lantao Liu. Autonomous navigation of AGVs in unknown cluttered environments: log-MPPI control strategy. IEEE Robotics and Automation Letters, 1(4):1024-11027, 2022.
[14] Anusha Nagabandi, Gregory Kahn, Ronald S Fearing, and Sergey Levine Neural network dynamics for mrodel-based deep reinforeement learning with model free fine-tuning, In 2018 IEEE International Conference on Robotics and Automation (ICRA). pages
IEEE, 2018.
[15] Alexander Pritzel, Benigno Uria, Sriram Srinivasan, Adria Puigdomenech Badia, Oriol Vinyals, Demis Hassabis, Daan Wierstra, and Charles Blundell. Neural episodic control. In International Conference on Machine Learning, pages 2827-2836. PMLR, 2017.
[16] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 2017.
[17] Shaoshuai Shi, Li Jiang, Dengxin Dai, and Bernt Schiele. Motion transformer with global intention localization and local movement refinement. In Advances in Neural Information Processing Systems, volume 35, pages 6531-6543, 2022.
[18] Yuval Tassa, Nicolas Mansard, and Emo Todorov. Control-limiter differential dynamic programming. In 2014 IEEE International Conference Robotics and Automation (ICRA), pages 1168 1175. IEEE, 2014.
[19] Evangelos Theodorou, Jonas Buchli, and Stefan Schaal. A generalized path integral control approach to reinforcement learning. The Journal of Machine Learning Research, 11:3137-3181 2010.
[20] Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In IEEE,
2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages
2012.
11

---

[21] Grady Williams, Andrew Aldrich, and Evangelos A Theodorou. Model predictive path integral control: From theory to parallel computation. Journal of Guidance, Control, and Dynamics, 4012):344-337, 2017.
[22] Haoru Xue, Chaoyi Pan, Zeji Yi, Guannan Qu, and Guanya Shi. Full-order sampling-hased MPC for torque-level locomotion control via diffusion-style annealing. In 2025 IEEE International Conference Robotics and Automation (ICRA), pages 4974-4981. IEEE, 2025.
[23] Zeji Yi, Chaoyi Pan, Guanqi He, Guannan Qu, and Guanya Shi. CoVO-MPC: Theoretical analysis of sampling-basse MPC and optimal covariance design. In 6th Annual Learning for Dynamics Control Conference, pages 1122 1135. PMLR, 2024.
[24] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint urXiv:1611.03530, 2016.
[25] Yihui Zuo, Mario Paolone, Fabrizio Sossan, Effect voltage source converters with electrochemical storage systems on dynamics of reduced-inertis -inettiia bulk power grids. Electric Power Systems Research, 189:106766, 2020.
12

---

A
Broader Impacts
Potential Field Theory has several potential societal implications that warrant thoughtful consideration. As framework that enhances control systems' ability to navigate complex environments, this technology could significantly improve reliability and safety in critical applications, including medical robotics, autonomous transportation, industrial automation, and disaster response systems. The demonstrated capabilities in disturbance recovery and obstacle avoidance could protect infrastructure during emergencies and reduce accident risks in human-maching environments. Additionally, the smoother control trajectories generated by our approach may contribute to energy efficiency and reduced mechanical wear, supporting sustainability efforts when deployed scale.
Beyond direct applications, our approach reduces the need for specialized domain knowledge and ex- tensive offline training compared to many reinforcement learning methods, potentially democratizing access to advanced control capabilities across a broader range of applications and organizations. This could create new opportunities for innovation in resource-constraing settings that cannot support extensive model training or system identification.
However, similar to many advances in automation, enhanced control capabilities could accelerate workforce transitions sectors relying on manual control operations. While likely creating new opportunities in system design and supervision, such transitions require thoughtful management to avoid disproportionate impacts on certain worker populations. Additionally, the improved navigation capabilities in complex environments could potentially be applied autonomous systems with dual- use concerns, including surveillance technologics or unmanned vehicles with security applications.
As control systems become increasingly capable, there's s also risk of overreliance leading to skill atrophy among human operators, potentially creating vulnerabilities during system failures when human intervention becomes necessary. The memory-based adaptation mechanism, while powerful, introduces additional complexity in understanding and predicting system behavior under novel conditions, which may create challenges for safety verification and accountability.
We believe these considerations should guide the development and deployment of memory-augment control systems. Strategies including open-source implementations, ples, interdisciplinary collaborations with ethicists and policy experts, educational initiatives. and engagement with appropriate regulatory frameworks can help maximize the positive impacts of this technology while mitigating potential risks. Our goal is to contribute to the responsible development of advanced control technologies that serve the broader social good while minimizing negative consequences.
centenee design princi-
B Detailed Memory Framework
B.1 Memory Representation Components
The memory representation M consists of elements representing significant topological features encountered during system execution. Each memory element (m4 T4s 76: k4,d,) contains five compo- nents that capture different aspects of critical features:
The feature position my E E" identifies locations in state space where the system encountered significant dynamics, typically areas where optimization became challenging. These positions mark critical points in the value function landscape.
The influence radius r E R++ defines the spatial extent around each feature position where the memory effect should be applied. This radius is determined adaptively based on the local geometry of the value function and observation system behavior near the critical point.
The strength parameter 7/2 E IR+ controls the magnitude of the memory feature's influence on the composite value function. This parameter evolves dynamically during execution according to the frequency and duration of system stagnation near the feature, with the update rule:
(1+1)
min (mmxx 7/ + AT), if [ mxIl S r; and stagnating
(19)
Baecuy;
if [are m,I > r: for > fhrreholdd
13

---

The type identifier H; {1,2,3} classifies features into three categories: 1) local minima, 2) low- gradient regions, or 3) high-curvature regions. This classification enables type-specific potential field designs tailored to the particular topological challenge each feature represents.
The direction vector d; E IR" provides guidance for optimization, particularly important for low- gradient regions where directional information helps the system overcome plateaus in the value function landscape. This vector is computed based historical escape directions that successfully navigated away from the problematic region.
B.2
Memory Update Mechanism
The memory update function I operates by detecting topological features through three primary mechanisms:
State stagnation detection identifies when the system state variance falls below threshold Ovar sliding window, indicating potential entrapment in a local minimum:
Var (Tt- w. 2t +1 24 < 6var
(20)
Gradient magnitude monitoring detects areas where the gradient norm of the value function falls below threshold 0 grdd signaling low-gradien regions:
Vasse (x:) < 0 graa
(21)
Curvature analysis identifies nigh-currature regions by examining the eigenvalues of the Hessian matrix:
Amnax V (Iie)
Oeury
(22)
Amin ( (22 base (xt))
When new feature is detected, it is added the memory representation it sufficiently distinct from existing features;
min iE (1.2......JIA [M]] || - 10tm-m1]] >
Oais
(23)
B.3 Potential Field Construction
The memory potential field is constructed weighted sum of basis potential functions, each tailored to address specific topological challenges:
M
Vinem (x, .)
= %s ((x, my, ri, Ks, d4)
Σ
(24)
The basis potential function & is selected based on the feature type K;:
Φ (z, mi, mi,Ti),
if Ki
6lx, mi, Fi, Ki, dc)
$2(2, mi, Ti; de),
if mi
(25)
Φ2 m,, Ty d4),
if K;
For local minima (N; = 1), we employ a repulsive function that generates outward forces, enabling escape from the local minimum basin:
|| (0.(1-11e m; 113
ti (I,, m4, ri) max
(26)
For low-gradient regions (N: = 2), we use directional function that provides guidance along previously successful escape directions:
(0, (0.(1-
| m, 112 (d. (x - m.))
(x, mi Tú, ds) = max
(27)
14

---

For high-curvature regions (K; = 3), we use saddle-like potential that facilitates navigation through narrow passages:
(2(2, ma, Ti, de) max (0, (0.1- || - ((d: (xr (ommm)))222 - d.dd )x: -
m;))
(28)
The adaptive weight function a(x; M) modulates the influence memory based on proximity to memorized features:
a(x, .) - a (s. min
|
mill
(29)
(E(II...|AMI)
where (2) 1 is the sigmoid function and > 0 controls the sharpness of the transition between nemory-dominater and base-dominated regions.
C Theoretical Proofs
C.1 Proof of Theorem 3.1 Non-convex Escape Property
To prove the non-convex escape property, we analyze the gradient of the combined value function V(x, within the local minimum region B(m,,r.).
Proof. For any point a E B(m;, the gradient of the value function in (5) can be:
VV(x, M) a(x, M)- Vhase + (1 ax, M)) V Vemem(r, M)
+ Va(x, M) (Vonse (2) Vimem (x, M))
(30)
Since B(m,, r:), we have || m4I S тs. and the dominant memory feature precisely the one at m,, For points near local minima, a(x, .M) becomes small due to the proximity to memory feature. Within ((m,,;,) r;), the gradient contribution from the memory term becomes
V (2, M) % V (z, m. r:)
(31)
For the repulsive potential Φ1, the gradient points outward from the center my;
(ol ri) = ?? (1
| mill?)
(x - -m:)
(32)
r?
Given that 7/ > 17 supre Mo,,):)) Vhasee the outward force from the memory term dominates the gradient of the base value function. The resulting effective gradient guides the system away from the local minimum.
Let Mour be the minimum outward gradient magnitude within B(m,, r4). The discrete-time dynamics with this outward gradient can be modeled as biased random walk with drift Mou and variance 2² from the system noise.
For such a process, the first-passage time T to exit the region has expectation bounded by E[T] < 2r (from standard results on biased random walks). By Markov's inequality, for any ó > 0:
Pont
E[T]
2r:
P(T > 1) S
(33)
Peut
Solving for the time needed to ensure P(T > 1) S 8, we get:
2r:
>
(34)
Mout 6
Therefore, we can set Tescape (8)
p establishing that the system will escape the local minimum
2r
region within finite time with probability at least - 6.
15

---

C.2 Proof of Theorem 3.2 Asymptotic Convergence Property
Proof. We first partition the state space I into regions: R& - B(m,, r:), the union of all memory feature regions, and Re = R" \ RRM the complement region.
In region Rc, the adaptive weight function arzz, MM) approaches as the distance to memory features increases. This means the value function behaves similarly to the base value function:
lim d(c,R M)
(x. .) =1
(35)
where d(x, RM) denotes the distance from x to set R.M-
By the coercivity assumption on base- for any bounded region B C R containing the global optimum xx, there exists finite time (6/22) such that:
P(31 S (8/2)) : x, E B) - 8/2
(36)
Once in the bounded region B, the system follows stochastic gradient process toward the global optimum. If B is chosen sufficiently small so that it contains no local minima of Vasse except x', and if B Ras 0, then standard stochastic approximation results guarantee convergence neighborhood of æ".
For any >> 0, there exists a time T2 8/2) such that
P
inf (2T2(16622)
|. x=1l S (TT E
6/2
(37)
By applying the law of total probability and noting that for any events A and B, P(A B)
P(A), we have:
P
inf 110e- 201-=x11- =x11(())) sc)
(38)
P
inf
|| x-11 sel (6/(6 EB) P (x e B)
(2.2(6.)/2)
2 (1 8/2) - (1))
5²
1 - 8 + 4
> 8-8
(39)
Defining the convergence time as Temn (e, (i(622 + T2(e, 6/2), we establish that
P
inf (2.m))))
|| - sc) 2 1-8
(40)
which completes the proof.
C.3 Proof of Theorem 3.3 Adaptive Learning Efficiency
Proof. Consider K independent local minimum regions L = {L1, Lz,..., probability that random trajectory enters region L,, and let T, be the expected time to escape from L, once entered.
Lx}. Let be the
For standard MPPI without memory, the expected total time spent in local minima regions can be expressed as
E E Haps =
N T
(41)
where N, is the expected number of times region L, is entered.
16

---

For random search process without memory, each local minimum can be encountered multiple times, and we can model N; as geometric random variable with parameter (1 9:), where qi the probability of returning to L after escaping. This gives EE[N]] - T-
For Memory-Augmented MPPI, after local minimum region is identified and added to memory, the probability of re-entering decreases significantly. Under ideal conditions:
E T
Trippps MA-MPPI
T
14 T
(42)
i=1j=2
where 7,isst the escape time on first encounter, and T? for j 2 2 are subsequent escape times (typically much shorter due to memory, i.., TV < Th
Since vi < 1 for j 2 2 (system rarely returns to memorized traps), we have:
K
K N. T. = E. MPPL
E
MPPP
TH
fist
(43)
The total expected convergence time consists of time spent in traps plus time spent in normal gradient search. For environments with significant local minima, the trap time dominates. The ratio of expected convergence times is approximately:
TMPPI
Ea No Ω(K) T
(44)
MPA-
This establishes that Memory-Augmentee MPPI provides an efficiency improvement that scales at least linearly with the number of local minimum regions in the state space.
D Connections to Related Mathematical Frameworks
The memory-augment augmented potential field framework has deep connections to several mathematical frameworks in optimization, differential geometry, and learning theory.
Memory-augmente potential fields can be viewed through the lens of Morse theory, which studies the relationship between the topology on it. For smooth function f _ + IR with non-degenerate critical points, the Morse index at critical point p is defined as the dimension of the negative eigenspace of the Hessian fp) Our memory mechanism effectively transforms local minima (index 0) into saddle points (index 2 1) or regular points, fundamentally altering the topological structure of the optimization landscape.
manifold and the critical points of smooth functions defined
From dynamical systems perspective, our approach modifies the vector field induced by the gradient of the value function. The base value function generates gradient flow 1 = = with attractors at local minima. The memory-augmente
Vhnae (z) system generates modified flow å = (x, M) where previously stable equilibria become unstable or are eliminated entirely, creating new flow patterns that guide the system away from problematic regions.
The adaptive weight function azx, M)) acts topological surgery operator, smoothly transitioning between the original value function and the memory-augment version. This creates composite manifold that preserves the global structure of the original optimization landscape while locally modifying its topology around critical features.
The construction of memory features bears resemblance the concept of persistent homology in topological data analysis, which studies how topological features persist across different scales. Our method dynamically identifies and preserves significant topological features (local minima, low-gradient regions) that impede optimization progress, effectively learning the persistent features of the value function landscape through system interaction.
E MA-MPPI Algorithm Details
MA-MPPI begins with an empty memory Mo = and nominal control sequence. each iteration, the controller detects topological features through three mechanisms: (1) state stagnation
17

---

detection when trajectory variance falls below threshold 6var, (2) gradient monitoring for regions where || (xse)))) 0 grad. and (3) curvature analysis examining Hessian eigenvalue structure. These detection mechanisms capture different aspects of challenging control landscapes that might impede optimization.
The memory update involves three operations: adding new features when encountering novel prob- lematic regions, merging similar features when their normalized distance falls below merga and dynamically adjusting feature strengths based on encounter frequency. The strength parameter evolves according (9.
(1+1) mimn(nnna Xmax Vi + Ay).
if 2 m.ll S ra and stagnating
Baccay:
if 2 m,ll > for >
Eshreshold
ensuring that frequently encountered obstacles gain prominence while rarely visited regions fade.
The enhanced value function combines the base task objective with memory potentials through an adaptive weighting mechanism that transitions smoothly between them based proximity to memorized features. The gradient of this enhanced function becomes Eq. (19):
(z,
axx, M,) base (I) +1 (x, Mc) V V.mem (I, M,)
+ Va(z, M,) (Vbase(x) Vmmm (z, M))
This gradient guides trajectory optimization, creating escape routes from local minima when combined with adaptive sampling.
The sampling process is enhanced through memory-based modifications to both temperature and distribution. When operating near memory features, the sampling covariance increases according to
En M.) = En,o (1 + p- (1 a(1,, M,))
(45)
enabling more aggressive exploration in challenging regions. For low-gradient regions with directional information, the sampling incorporates bias along stored direction vectors.
The control sequence optimization follows the path integral formulation, where the optimal control the expectation over weighted samples:
ui -E((l))lur(T))
Wk
ut
(46)
== Wi
with weights wy exp(- X, )) determined by trajectory costs and adaptive temperature.
This integration of memory-based value function enhancement with adaptive sampling creates control system that effectively navigates complex environments by learning from experience. The dynamic memory representation continuously evolves based on system interactions, enabling increasingly efficient navigation through challenging control landscapes.
F Topological Feature Detection Details
F.1 Detection Mechanisms
MA-MPPI employs three complementary detection mechanisms to identify topological features that impact optimization performance.
State stagnation detection identifies local minima by calculating the statistical variance of states within a fixed time window:
Σ Var(X,) = K i= K+1
c - 12
(47)
where X, contains the most recent K states, and Ty is the average state within the window. When this variance falls below threshold Oyar. the system considered stagnant, typically indicating entrapment in local minimum. The appropriate value of Ovar depends the characteristic scale of the state
18

---

space and is typically set to Pvar = 0.01 - o where of represents the expected state variance during normal operation.
Gradient analysis examines both magnitude and directional properties of the value function gradient. For magnitude analysis, the system computes V (dre)l and identifies low-gradient regions when this value falls below threshold ggrad For directional analysis, the system calculates the angle change between consecutive gradient vectors:
(cos | Vaase (24) V base (Tc-
Vinase (2+4 V Vusse (It-
Ad;
(48)
When Ad, exceeds the threshold Oeurv. the region is identified as a high-curvature area requiring special attention.
Curvature analysis provides a more comprehensive understanding the local landscape geometry by examining the eigenvalue structure of the Hessian matrix (ase Specifically, the system calculates the condition number:
Amax
A V Viase (Ia))
(49)
Amin
where Amax and Amin the maximum and minimum cigenvalues of the Hessian. High condition numbers indicate regions with significant anisotropy, such as narrow valleys or ridges.
F.2
Feature Classification and Representation
Fach detected feature is classified into one of three types based on the detection mechanism that identified it:
Type 1 (Local Minima): Identified primarily through state stagnation, these features represent regions where the controller becomes trapped. They are characterized by low state variance and persistent inability to make progress despite control effort.
Type 2 LLo-Grradien Regions): Identified through gradient magnitude analysis, these features represent plateaus in the value function landscape. They are characterized by gradient magnitudes below threshold yrd despite the system not being true minimum.
Type (High-Curvature Regions): Identified through gradient direction changes or Hessian analysis, these features represent areas with challenging geometric properties such narrow valleys, sharp ridges, or saddle points.
Each feature is represented as a tuple (m; Ta Kiy de) where:
m, E R is the feature position state space
Fz R + is the influence radius defining the feature's spatial extent
% R is the strength parameter indicating importance
E (1, 3) is the type identifier
d; E R= the direction vector (for types and 3) providing guidance information
The classification determines which potential function is applied in the memory- value function, with each type receiving a specially designed potential to address its particular challenges.
F.3
Feature Consolidation
To maintain computational efficiency, MA-MPPI employs feature clustering and merging mechanisms. When newly detected feature point meew is spatially close to existing features of the same type, a merging operation is performed according to the following criteria:
Merge if: ||mmnw - mlll
Omerge and Kneww Ki
(50)
where Omerge is typically set to 1.5, representing significant overlap between feature influence regions.
19

---

When merging features, the system computes weighted averages for position and influence radius:
7/
+ 'new 7/lnew
Mmmerged
(51)
Y/' + Vnew
1| Mpowl]
'mergged
max
Fs: neew:
+ min(ri, (new
(52)
The strength parameter is accumulated to reflect the combined importance:
Tinerged
.
"/ne
(53)
For features with direction vectors (types 2 and 3), the merged direction is computed as a weighted average, then normalized:
Y . de + Vnew dnew
dmeergod
(54)
1/4 di + Ynew dnew
F.4
Dynamic Feature Strength Update
Feature strength parameters evolve dynamically based on system experience. The update rule follows:
min (7mxx,
(e) + A7),
|| late mz|l S r; and Var(X.) < Ovar
(++1) %
Baccay,
if || - m: r; for > fihreboodd
(55)
otherwise
Here, 7max is the maximum allowable strength (typically 5.0), By is the increment per stagnation event (typically 0.1), Bassay the decay factor (typically 0.99), and fhreshold is the time period after which decay begins (lypically 100 time steps).
When feature's 's strength falls below the removal threshold Ymin (typically 0.1), it is removed from the memory representation. This ensures that only persistently relevant features are maintained, while those that become obsolete gradually fade away.
F5 Implementation Considerations
The detection mechanisms operate at different time scales to balance computational efficiency with detection accuracy. State stagnation analysis occurs continuously, it directly impacts control performance. Gradient analysis is performed at regular intervals (typically every 5-10 control steps) to identify developing problematic regions before they cause stagnation. Curvature analysis, being more computationally intensive, is performed selectively when gradient analysis indicates potential high-curvature regions, periodically at longer intervals (typically every 20-30 control steps).
To manage computational complexity high-dimensional state spaces, dimensionality reduction tech- niques can be applied before feature detection. Principal Component Analysis (PCA) or task-relevant subspace identification methods help focus detection on the most critical dimensions. Additionally, incremental detection algorithms allow the system to update feature estimates progressively rather than recomputing them from scratch at each step.
The detection thresholds Ovar, Ograd, and Ocury can be adaptively tuned based on observed system performance. During initial operation, more conservative thresholds ensure that important features are not missed. As the system accumulates experience, thresholds can be adjusted to focus on the most significant features, improving computational efficiency while maintaining detection of critical obstacles.
G Adaptive Potential Field Synthesis Details
G.1 Feature-Specific Potential Functions
MA-MPPI employs type-specific potential functions to address different topological challenges. For each feature fi = (m,, ra, 7h, 6s; d;) in the memory representation, the contribution to the memory potential depends its type K;:
20

---

Type 1: Local Minimum Features These employ radially decreasing function that creates repulsive potential field:
&1 (1, max (1-ll--ma)
(1-11_1-= m.
(56)
This function reaches its maximum at the feature center and smoothly decreases to zero at the boundary the influence radius. The resulting gradient creates a repulsive force pushing the system away from known trap regions.
Type 2: Low-Gradient Features cues:
These use directional guiding function that provides navigation
(2 (4, d4) max 0, 11-Il=-mal
m..1²) (1
(d: (a m.))
(57)
where d; is the preferred direction vector. This function creates asymmetric potential field that guides the system along favorable directions through problematic regions.
Type 3:
Cugh-Curvature Features complex terrain:
These employ saddle-point function that helps navigate
(0((1-1e-m_l))))
mill?)
[d; (x mz))²
3) 8))
Φ (2 x. m,, rs. d.) max 0.
(58)
||
m: 1|
where 3 E (0. 1) controls the saddle shape. This function creates channels through high-curvature regions, enabling traversal of narrow corridors mountain passes in the potential landscape.
G.2 Memory Potential Construction
The complete memory potential combines contributions from all features:
M
Vinem (r. M,)
Ta Dr, (x, mi, Fi, di)
(59)
where "/ is the strength parameter indicating the feature's importance. This formulation ensures that more frequently encountered or persistent challenges have stronger influence on the enhanced value function.
G.3 Adaptive Weighting Mechanism
The weighting function a(x, M,) balances base and memory potentials:
d0
a<x, M,) min 1, 8(x, M,) +
(60)
where 6(2, M,) measures proximity to memory features:
8(z, M,) = Σ Ta max (,,
||x m4
(61)
5y is a scaling parameter (typically 0.5), and e is a small positive constant preventing division by zero This formulation ensures smooth transition between navigation modes: when far from memory features, a and the system follows the base objective; when near significant features, a and memory guidance dominates.
G.4 Temperature Adaptation Mechanism
The adaptive temperature parameter adjusts sampling exploration:
X(x, M,) Ao (1 + n (1
(62)
21

---

where Åo is the base temperature (typically 1.0) and is the enhancement coefficient (typically 2.0-5.0).
This mechanism increases exploration specifically in challenging regions, with the sampling distribu- tion variance scaling proportionally:
X(x, M,)
(x, M.) Eu,0
(63)
A
where M0,0 is the nominal control sampling covariance.
The dual-layer adaptation (value function + temperature) creates synergistic effect: the modified value function shapes the landscape avoid problematic regions, while increased exploration helps discover alternative paths that might not be apparent in the base potential.
G.5 Computational Optimizations
Several optimizations ensure efficient implementation:
Sparse Computation Feature potentials are only evaluated for features whose influence regions contain the query point:
6R. (I. mi, ri; da) = 0 if || - mill > ri
(64)
This reduces computation when the memory contains many features.
Spatially-Indexxe Memory Features organized in spatial data structure (typically k-d tree), enabling efficient querying of relevant features for any state point.
Gradient Caching When computing trajectories, gradients of the enhanced value function cached and reused when evaluating nearby states, reducing redundant computation.
Approximate Field Synthesis For very large memory representations, the potential field can be pre-computed on a grid and interpolated, trading accuracy for speed in scenarios where the memory contains hundreds of features.
These optimizations ensure that the memory enhancement mechanism maintains real-time perfor- mance even the memory grows with system experience. In practice, the computational overhead remains minimal compared to the MPPI sampling process, typically adding less than 10% to the overall computation time.
H Additional Details of Robotic Control Experiments
H.1 Experimental Details
H.1.1 Environment Specifications
We selected four representative control environments from OpenAI Gym and MuJoCo 20
Pendulum-v1 A classic control task involving swinging a pendulum from a random initial position to an upright position and balancing it there. The challenge arises from limited torque and nonlinear dynamics. The state space is 3-dimensional (angle, angular velocity), and the action space 1-dimensional (torque).
BipkdalWalker-v3 A 2D walking simulation that requires controlling robot to traverse terrain with obstacles. The state space is 24-dimensional (position, velocity, joint angles, etc.), and the action space is 4-dimensional (joint torques). The reward function encourages forward progress while penalizing energy consumption and harsh landings.
HalfCheetah-v4 A MuloCo-based 2D running robot simulation with 6 rotational joints. The state space 17-dimensional (position, velocity, joint angles), and the action space is 6-dimensional (joint torques). The reward function encourages forward velocity while minimizing control effort.
22

---

Humanoid-v4 Our most complex task, featuring 3D humanoid robot with 17 joints. The state space is rro-dimensional (position, velocity, joint angles, contact forces), and the action space is 17-dimensional (joint torques). The reward function encourages maintaining an upright posture and forward motion while minimizing energy consumption.
H.1.2 Implementation Details
MA-MPPI Configuration For MA-MPPI implementation, we used environment-appronn diction horizons: 15 steps for Pendulum-vi, 20 steps Bipeda/Walker-v3. 25 steps for HalfCheetah- v4, and 35 steps for Humanoid-v4, reflecting the increasing dynamics complexity. The memory module was configured to store up 100 topological features, with the feature strength decay factor set to 0.95. The feature detection thresholds were tuned for each environment: state stagnation threshold Orar = [0.01, 0.05, 0.03, 0.08), gradient threshold Oprad = [0.005, 0.01, 0.01, 0.02]. and curvature threshold Peurv [0.5, 0.4, 0.6, 0.8] for Pendulum-vi. v4, and Humanoid-V v4 respectively, where the values brackets indicate the environment-speciffe speciffc parameters listed in the same order as the environments. The MPPI temperature parameter set to A = 0.1 with enhancement coefficient 2. For all experiments, we used 1000 samples per control iteration.
pre-
SipeealWalkeer-ee HalfCheetah-
Baseline Configurations Standard MPPI was implemented with identical sampling parameters but without memory augmentation. For reinforcement learning baselines, we used implementations from Stable Baselines3 with the following configurations:
SAC: Twin Q-networks with automatic entropy tuning, learning rates of 3 x 10-4, batch size of 256, and replay buffer size of 1,000,000.
PPO: GAE-À with À = 0.95, value function coefficient of 0.5, entropy coefficient of 0.01, and learning rate of 10-4
DDPG: -eiieeinn-Uhlnbbekkkkk noise with 0.2, learning rates of x 10-3 for critic and 10 for actor, and replay buffer size 1,000,000.
For traditional optimal control baselines, we used the following configurations:
iLQR: Regularization parameter of 1e-6, line search parameter of 0.5, and convergence threshold of le-6.
MPC: Black-box dynamics model trained with a neural network, using a 5-step history for prediction, batch size of 128, and learning rate of le-3.
Computational Resources All experiments were conducted on a computing cluster with Intel Core 19-12900K CPUs (3.20GHz) and NVIDIA RTX 3070 GPUs. The MPPI-based methods were implemented in Python with JAX for GPU acceleration. The reinforcement learning basclines utilized PyTorch.
H.1.3 Evaluation Metrics
We employed properly defined metrics to capture both adaptation efficiency and asymptotic perfor- mance:
Sample Efficiency (Nsu%) Defined as the minimal number of environment interactions required to reach 80% of asymptotic performance:
Ns0% min(n: R(n) 2 0.8 Rayymp)
(65)
where R(n) is the average reward after n E * interactions and Rasymp is the average reward during the stability phase.
Cumulative Reward (Reum)
Represents the total reward in an episode:
T
Reum
r(s,, ae)
(66)
1=0
23

---

Local Optima Escape Rate (P escape) Quantifies the controller's ability to escape from predefined trap states. We define a trap state Stap as any state from which the expected return falls below
a Vmax (where 0.5 and Vmx is the maximum achievable value) and the agent remains within neighborhood. N (Stra) for at least Threshold 50 time steps without improvement. The escape rate
Nescape 100%
Pescpe
(67)
Nutias
where Nescupe is the number of successful escapes from intentionally initialized trap states and Nuials the total number of evaluation trials starting from predefined trap states.
Trap Frequency (Firap)
Measures how often method becomes trapped during normal operation:
Nstuck 100%
F Fura
(68)
Nepisodes
where Nauck counts episodes containing identified trap states and Nepisodes the total number of evaluation episodes conducted.
Computational Efficiency Assessed through average computation time per control step:
N
T comp
(69)
Value Consistency (pv) Evaluates prediction accuracy:
PV Corr(Vpreat: Ractual)
(70)
where Vored is the value predicted by the controller, Racuual is the actual discounted return, and Corr represents the Pearson correlation coefficient measuring the linear correlation between the two variables.
H.2 Sample Efficiency Analysis
Table presents the complete sample efficiency comparison, showing the number of environment interactions required to reach 80% of asymptotic performance for each method and environment.
Table Complete sample efficiency comparison: Environment interactions required reach 80% of
asymptotic performance.
Method
Pendulum-vi
BipedalWalker-v3
HalfChectah-v4
Humanoid-ve v4
MA-MPPI (Ours)
72+
183+-24
424
568+173
MPPI
124+18
352::41
5861168
984=127
SAC
2671332
6241:58
1248 106
1875+4215
PPO
312:141
7361-67
1456
2105++273
DDPG
293:1:7
684+75
1368=142
1953+:246
iLQR
185+23
426+-52
1188+131
1730++201
MPC
156+119
387+46
753
1236+1166
The sample efficiency advantage of MA-MPPI is particularly notable in more complex environments. For Humanoid-v4, MA-MPPI requires only 568 interactions to reach 80% performance, compared to 1875 for SAC (3.3x improvement) and 984 for standard MPPI (1.7x improvement). This pattern suggests that the memory mechanism becomes increasingly valuable as task complexity increases, providing the greatest benefit in high-dimensional, highly non-convex control problems.
H.3 Trap Frequency Analysis
Table presents the trap frequency for each method across all environments, showing how often each controller becomes trapped during normal operation.

---

Table 6: Trap frequency analysis: Percentage of episodes containing trap states.
Method
Pendulum-vl
BipedalWalker-v3
HalfChectah-v4
Humanoid-v4
MA-MPPI (Ours)
1.2::0.4
1.7:0..5
2.3
0.9
MPPI
3.8::..7
4.6:10.9
6.2
8.1
SAC
2.5.:0.5
3.2
4.1.1.0
1.3
PPO
2.9:0.66
3.7
4.8
6,2+1.6
DDPG
3.340.6
4.1
5.4
7.3
iQQR
5,240.9
6,8
8.7
11.422.4
MPC
3.5
4.3
5.8
7.6
The trap frequency results demonstrate MA-MPPI's ability to proactively avoid problematic states based on past experience. Humanoid-v4, MA-MPPI encounters trap states 2.9x less frequently than standard MPPI and 1.9x less frequently than SAC. This proactive avoidance translates directly better real-worle performance, robots using MA-MPPI spend significantly less time recovery behaviors and more time making progress toward goals.
H.4 Ablation Study Results
Table presents the complete ablation study, showing the performance impact when removing different components of MA-MPPI.
Table 7: Complete ablation study: Performance impact when removing components (% decrease from full MA-MPPI).
Configuration
Pendulum-vi
HalfCheetah--ee BipkdalWalker-v3
Humanoid-ve
Full MA-MPPI
0.0%
0.0%
0.0%
0.0%
No adaptive weights
18.3%
20.6%
22.4%
25.1%
No feature detection
31.5%
34.2%
37.6%
40.3%
No memory module
42.7%
46.5%
52.3%
58.1%
The ablation results reveal the relative importance of each component:
The memory module provides the largest contribution, with removal causing a 42.7-58.1 1% performance decrease
Feature detection is the second most important component (31.5-40.3% decrease when removed)
Adaptive weights provide significant but smaller contribution (18.3-25.1 1% decrease)
The increasing impact of all components with environment complexity indicates that MA-MPPI's design particularly well-suited for challenging control problems, with each component contributing more significantly as the task becomes more difficult.
H.S Computational Overhead Analysis
Table g presents the average computation time per control step for each method and environment, along with the percentage increase relative standard MPPI.
MA-MPPI introduces a modest computational overhead compared to standard MPPI, ranging from 12.0% for Pendulum-vl to 18.2% for Humanoid-v4. This overhead includes the cost of feature detection, memory updates, and enhanced value function computation. The increasing overhead with environment complexity reflects the growing memory size and more complex feature interactions in higher-dimensional spaces.
We also observed that MA-MPPI tends produce smoother control trajectories with fewer high- frequency oscillations compared standard MPPI. Analysis of control signal frequency content showed 24-37% reduction in high-frequency components across environments. This smoother
25

---

Table 8: Computational overhead analysis: Average computation time per control step (ms).
Method
Pendulum-vl
BipedalWalker-v3
HalfChectah-v4
Humanoid-v4
MA-MPPI
8.4 (+12.0%)
12.7 (+14.4%)
18.3 (+16.5%)
24.6 (+18.2%)
MPPI
7.5
11.1
15.7
20.8
SAC (inference)
1.2
2.3
3.8
5.7
PPO (inference)
1.1
2.1
3.5
5.2
DDPG (inference)
1.2
2.2
3.7
5,6
iQQR
6.3
9.8
14.2
19.5
MPC
10.2
15.9
22.7
31.4
control behavior particularly beneficial for physical robot systems, where rapid oscillatory control cause mechanical wear, energy inefficiency, and undesirable dynamics.
It's worth noting that while RL methods have faster inference times, they require extensive offline training that isn't t captured in these measurements. MA-MPPI's online learning approach elimi- nates this offline training requirement, making more suitable for deployment new or changing environments.
H.6
Memory Size and Feature Type Analysis
Table 9 presents the average memory size (number of stored features) at different time points during operation for each environment.
Table 9: Memory size analysis: Average number of stored features different time points.
Time Point
Pendulum-vl
BipedalWalker-v3
HalfCheetah-vä
Humanoid-v4
500 steps
8.3
12.6#:2.1
17.4+:2.8
23.1#3..7
1000 steps
14.5+1.8
21.2+2.77
29.6+
38.4#4.66
1500 steps
17.2+2.0
26.8+3.1
35.7+3.9
45.2+5.2
2000 steps
18.542.1
28.3#332
38.1+4.1
48.6=5.5
Table I0 presents the distribution of feature types detected in each environment, showing the percent- age of features classified as local minima (Type 1), low-gradient regions (Type 2), and high-curvature regions (Type 3).
Table 10: Feature type distribution: Percentage (%) of each feature type detected.
Environment
Type (Local Minima)
Type (Low Gradient)
Type 3 (High Curvature)
Pendulum-vl
52.3%
28.6%
19.1%
BipedalWalkee v3lker-v3
44.7%
32.5%
22.8%
HalfCheetah- v4 Humanoid-và
38.2%
35.8%
26.0%
35.6%
37.2%
27.2%
The memory size increases rapidly during initial exploration, then stabilizes feature consolidation and forgetting mechanisms balance new feature detection. The final memory size scales with environment complexity, reflecting the greater number of challenging regions in higher-dimensional control problems.
The feature type distribution reveals an interesting pattern: simpler euvironments like Pendulum-v/ have more distinct local minima (Type I features), while complex environments like Humanoid-v4 have more balanced distribution with higher proportions of low-gradient (Type 2) and high-curvature (Type 3) regions. This aligns with the intuition that high-dimensional control problems tend have more plateaus and saddle points rather than clear local minima.
26

---

I Additional Details of Complex Engineering Systems Experiments
I.1 Evaluation Methods and Metrics
We comprehensively evaluated our method against several state-of-the-art baselines, each carefully configured to ensure fair comparison:
Standard MPPI 21:
Implemented following the original formulation with an exponentially weighted averaging scheme over sampled trajectories. For UAV experiments, we used 1000 trajectory samples with an optimization horizon of 25 steps and a temperature parameter
À = 0.1. For power systems, we used 800 samples with 30-step horizon. Control limits were enforced through sigmoid transformation of unbounded control samples.
Diffusion Policy [3 Implemented using a conditional diffusion model with 8 denoising steps. The architecture consisted of UNet backbone with 4 residual blocks, dropout rate 0.1, and channel widths of [128, 256, 512, 1024]. For UAV control, we used a context window of 10 historical states, while power system experiments used 15 historical states. The model was trained on 100,000 demonstration trajectories collected from a mixture of expert controllers for 500,000 gradient steps using the Adam optimizer with learning rate of 3e-4 and batch size 256.
Motion Transformer [17: We employed an encoder-decoder architecture with 6 trans- former layers, 8 attention heads, and an embedding dimension of 512. For UAV experiments, positional encodings incorporated obstacle information, while power system experiments used frequency and voltage measurements as key variables for attention. The model was trained using teacher forcing with a cross-entropy loss for 300,000 gradient steps using Adam optimizer with weight decay le-4, learning rate le-4, and batch size 128.
MLP-based MPC [14]: We implemented a neural network dynamics model consisting of 4 hidden layers [1024, 512. 256, 128] with ReLU activations. The model trained to predict next-state deltas using a dataset of 200,000 state-action-state transitions collected from system interaction. Training used the MSE loss with the Adam optimizer, learning rate 1e-3, batch size 256, and ran for 100,000 gradient steps with early stopping based on validation error. The dynamics model was integrated with sceding-horizzl controller using CEM (Cross-Entropy Method) optimization with 500 samples per iteration and 5 iterations.
DKO-based MPC [12 6]: The Deep Koopman Operator approach used an encoder-decoder architecture with hidden layers [256, 512, 256] for embedding states into a 128- 28-dimensional linear latent space. The Koopman operator was represented as 128x128 28 matrix learned jointly with the encoder-decoder networks. For UAV control, we used lifting dimension of 128, while power systems used 256 dimensions to capture the more complex dynamics. Training employed a composite loss function combining reconstruction error, prediction error, and linearity constraints with weights [0.4, 0.4, 0.2]. The model was trained on 150,000 transitions for 200,000 steps using RMSProp optimizer with a learning rate of 5e-4 and batch size of 128.
In all experiments, we conducted 30 independent trials with different random seeds to ensure statistical robustness. Statistical significance assessed using paired t-tests with Bonferroni correction for multiple comparisons, establishing significance at p < 0.01. We ensured all methods had access to identical system information and operated under the same control frequency constraints.
Performance metrics were carefully selected to provide a comprehensive evaluation across multiple dimensions:
Task success:
Binary success/failure metrics specific each domain
Solution optimality: Quantitative measures of solution quality relative to the theoretical optimum
Control smoothness: L.2 norm of control acceleration and jerk assess motion quality
Computational efficiency: Wall-clock time per control step, measured on identical hardware
Local minima escape capability: Success rate when initialized challenging configura- tions
27

---

All experiments were conducted on homogeneous computing cluster with Intel Core 19-12900K CPUs (3.2GHz, 16 cores) and NVIDIA RTX 3070 GPUs (&GB VRAM). Implementation used JAX 0.4.1 for GPU acceleration of compling-based computations, with PyTorch 1.13.1 for neural network baselines. To ensure reproducibility, we used fixed random seeds for environment initialization while maintaining separate seeds for controller stochasticity.
1.2
UAV Obstacle Avoidance Experimental Setup
The UAV experiments featured 20 distinct navigation scenarios with increasing complexity, from simple obstacle arrangements to maze-like environments with narrow passages, dead-ends, and multiple possible routes.
The UAV had a mass of 1.5kg and maximum thrust of 30N, with a diagonal inertia matrix J = d0.g(0.0125,, 0.0125, 0.0225) kg.m². The drag coefficient was set to ka 0.1. Control inputs were constrained to f E [0, 30) N and E 0.5, 0.5) Nm.
For the MA-MPPI configuration, we used prediction horizon of 25 steps with 1000 trajectory samples per control iteration. The memory module was configured to store up to 50 topological features with detection thresholds Opar = 0.05, 0 0grad 0.01, and O Oeure- = 0.4. The feature strength decay factor was set 0.95.
1.3
UAV Obstacle Avoidance Detailed Analysis
I.3.1
Success Rate Analysis
The success rate advantage of MA-MPPI (94.3% vs. 72.8% for standard MPPI) increased with environment complexity. In the most challenging scenarios with multiple narrow passages, MA- MPPI maintained an 89.5% success rate while standard MPPI dropped to 58.7%.
Success rate improvements stemmed from three key capabilities:
Proactive avoidance of previously identified trap regions
Adaptive sampling strategy that focused exploration where needed
Accumulating knowledge of environment topology over time
We observed that after encountering particular obstacle configuration once, MA-MPPI significantly improved its performance when facing similar configurations again, demonstrating effective learning from experience.
I.3.2 Local Minima Escape Analysis
The most significant advantage MA-MPPI was local minima escape capability (87.5% vs. 34.2% for standard MPPI). We further analyzed this capability in three specific trap scenarios:
Corner traps: MA-MPPI escaped in 91.3% of trials (vs. 42.8% for standard MPPI)
U-shaped obstacles: MA-MPPI achieved an 84.7% escape rate (vs. 31.5% for standard MPPI)
Narrow corridors: MA-MPPI successfully navigated through in 86.2% of trials (vs. 27.6% for standard MPPI)
Analysis of trajectory data revealed that MA-MPPI's memory- based potential fields created "tunnels" through challenging regions, guiding sampling toward promising escape routes. Additionally, the adaptive temperature mechanism increased exploration specifically in identified trap regions, enabling discovery of solutions that standard methods missed.
1.3.3 Computational Efficiency Analysis
MA-MPPI introduced a 23.5% computational overhead compared to standard MPPI (12.6ms vs. 10. 1m.2ms per control step). This overhead consisted of:
Feature detection and memory updates: 1. (8.7%)
28

---

potential field computation: 0.8ms (6.3%)
Enhanced trajectory evaluation: 0.5ms (4.0%)
Despite this overhead. MA-MPPI remained more computationally efficient than learning- based approaches (Diffusion Policy: 18.4ms, Motion Transformer: 22.7ms). Crucially, the computation time stabilized after approximately 500 time steps as the memory consolidation mechanisms balanced new feature detection with forgetting of less important features.
1.4 Power System Control Experimental Setup
We used the IEEE 39-bus New England test system with 10 generators and 39 buses. The system operates at a nominal frequency of 60Hz with base power of 100MVA. The three critical disturbances were:
Three-phase fault on the line connecting buses 16-17, cleared after 150ms
Load increase of 25% at buses 4, 12, and 20, ramping over 200ms
Trip of the 650MW generator at bus 33, with instantancous power loss
The power system was simulated using variable-step solver with maximum step size of 10ms. Control inputs included generator output adjustments, transformer tap settings, and capacitor bank switching operations.
For the MA-MPPI configuration, we used a prediction horizon of 30 steps with 800 trajectory samples. The memory module stored up to 75 features with detection thresholds f.ar 0.08, ggrad = 0.02, and deree 0.6. The control interval was 100ms for normal operation, automatically reducing to 50ms during detected disturbances.
1,5 Power System Control - Detailed Analysis
L5.1 Constraint Violation Analysis
MA-MPPI achieved 59.6% reduction in constraint violations compared to standard MPPI (2.3% vs. 5.7%). This advantage most pronounced during the three- three-phase fault scenario, where MA-MPPI kept violations to 4.7% compared to 12.3% for standard MPPI.
Analysis by constraint type revealed:
Voltage violations: MA-MPPI 1.8% vs. standard MPPI 6.2% (71.0% reduction)
Thermal violations: MA-MPPI 4.3% vs. standard MPPI 7.8% (44.9% reduction)
Stability violations: MA-MPPI 0.7% vs. standard MPPI 4.1% 1% (82.9% reduction)
The memory mechanism effectively identified regions of state space associated with constraint violations, creating repulsive potential fields that guided the controller away from these regions preemptively.
1.5.2 Disturbance Recovery Analysis
MA-MPPI achieved 51.7% faster recovery from disturbances compared to standard MPPI (4.2s vs. 8.7s). Recovery performance by disturbance type:
Three-phase fault: MA-MPPI 4.8s vs. standard MPPI 8.3s (42.2% faster)
Load change: MA-MPPI 4.1s vs. standard MPPI 8.4s (51.2% faster)
Generator trip: MA-MPPI 3.7s vs. standard MPPI 9.4g 4s (60.6% faster)
The most significant advantage was observed the generator trip scenario, where MA-MPPI quickly redistributed power flows while maintaining frequency stability. The memory memory-augmenneed controller effectively learned patterns from previous disturbances, enabling it to recognize and respond to similar situations more effectively over time.
29

---

The consistency in performance was particularly notable, with a standard deviation in recovery time 0.6s for MA-MPPI versus 1.9s for standard MPPI, indicating more reliable and predictable disturbance responses.
J Hyperparameter Sensitivity Analysis
To assess the robustness and practicality of the MA-MPPI algorithm, we conducted a comprehensive sensitivity analysis key hyperparameters. This analysis is crucial for understanding the algorithm's 's adaptability in different scenarios and providing parameter tuning guidelines for practical applications.
J.1
Key Hyperparameters and Testing Methodology
We identified four groups of key hyperparameters that significantly influence MA-MPPI performance:
Feature detection thresholds:
Ovar (state stagnation detection), Ograd (gradient magnitude monitoring), and Ocurv (curvature analysis)
2. Memory feature parameters: and decay factor Sdecey
influence radius multiplier Ky, initial feature strength '7..
3. Adaptive weight parameters: scaling parameter 6, and transition sharpness parameter 3 in the balancing function
4. Temperature adaptation parameters: base temperature Àg and enhancement coefficient
For each parameter, we varied its value by =50% around the central value while keeping other pa- rameters fixed, testing performance on the Humanoid-va environment. Each parameter configuration evaluated through 30 independent experiments, recording success rate, cumulative reward, and local minima escape rate metrics.
J.2 Feature Detection Threshold Sensitivity
Feature detection thresholds determine the sensitivity of the system in identifying and memorizing problematic regions.
Table 11: Performance change (%) relative to baseline for different feature detection thresholds.
Parameter variation
Ovar
0 Ograd
Ocura
-50%
-4.2%
-12.7%
-3.8%
-25%
-1.7%
-5.8%
-1.5%
Baseline
0.0%
0.0%
0.0%
+25%
-2.3%
-6.4%
-2.1%
+50%
-7.6%
-15.3%
-5.2%
The data indicates that the gradient detection threshold 0 yrad has the largest impact on performance, with +50% variations causing 12.7-15.3% performance decrease. This is because gradient infor- mation directly influences the choice of escape directions. In contrast, the state stagnation threshold Ovor and curvature threshold Ocuro have smaller impacts, indicating algorithm robustness these two parameters.
Notably, all parameters exhibit an inverted U-shaped sensitivity curve-tow too low thresholds lead to overly sensitive detection (memorizing too many unimportant features), while too high thresholds result in missing important features. Within the +25% variation range, performance decreases are typically contained below 6%, demonstrating reasonable parameter tolerance for the algorithm.
J.3 Memory Feature Parameter Sensitivity
Memory feature parameters control how memory elements influence the value function landscape.
Results show that the influence radius multiplier is the most sensitive parameter, especially when set too small. Insufficient influence radii prevent memory features from effectively altering the value function landscape across adequate range, causing the system to still potentially fall into local
30

---

Table 12: Performance change (%) relative to baseline for different memory feature parameters.
Parameter variation
Influence radius Kr
Initial strength 70
Decay factor Bdecay
-50%
-1.44%
-7.3%
-3.2%
-25%
-8.7%
-3.5%
-1.4%
Baseline
0.0%
0.0%
0.0%
+25%
-2.3%
-2.8%
-2. %
+50%
-9.1%
-8.5%
-5.7%
minima. In contrast, initial strength no and decay factor Saeeag have more moderate impacts and exhibit stability across wide range of values.
Particularly noteworthy that increasing the influence radius (+25%) actually slightly improved performance, suggesting that the default setting may be too conservative in some environments. This provides a potential tuning direction.
J.4 Adaptive Weight and Temperature Parameter Sensitivity
These two parameter groups control the balance between memory potential fields and base objectives, as well as the exploration aspect of the sampling strategy. Table summarizes the sensitivity test results.
Table 13: Performance change (%) relative to baseline for adaptive weight and temperature parame- ters.
Parameter
-50%
-25%
Baseline
+25%
+500%
Scaling parameter do
-1.33%
-4.8%
0.0%
-3.2%
-8.5%
Transition sharpness 3
-4.2%
-1.7%
0.0%
-2.4%
-6.1%
Base temperature Ào
-13.7%
-5.9%
0.0%
-7.3%
-16.4%
Enhancement coefficient y
-9.8%
-4.1%
0.0%
+1.2%
-4.7%
The base temperature λy shows the highest sensitivity, which is expected as directly controls the peploration-exploitaaioo trade-off. Too low temperature leads to premature convergence suboptimal solutions, while too high temperature results in excessive exploration and noisy control signals.
Interestingly, the enhancement coefficient 1/ at +25% actually improved performance, suggesting that increased exploration when facing memory features can be beneficial complex environments. The scaling parameter &g has moderate sensitivity, while the transition sharpness /3 has relatively minor impact.
J.5 Relationship Between Environment Complexity and Parameter Sensitivity
We further investigated how environmental complexity affects parameter sensitivity.
Table 14: Performance change (%) for 0 grad variations across different environments
Environment
0 gred -25%
fgrad +25%
Pendulum-vi
-1.3%
-1.8%
BipedalWalkee valker-33
-2.7%
-3.4%
HalfCheetahhvä
-4.2%
-5.1%
Humanoid-và
-5.8%
-6.4%
Results indicate that parameter sensitivity increases with environment complexity. This is because high-dimensional state spaces and complex dynamics make precise detection of topological features more critical. Nevertheless, even in the most complex Humanoid-v4 environment, +-25% parameter variations only lead to approximately 6% performance degradation, attesting to the algorithm's robustness.
31

---

J.6 Optimal Parameter Ranges and Practical Guidelines
Based on our sensitivity analysis, we recommend the following parameter ranges for environments of varying complexity:
Simple environments (e.g., Pendulum):
Over E [0.008, 0.015), Ograd E [0.004, 0.007], Ouroo E [0.4, 0.7]
Ky E [1.8, 2.5], An E [0.08, 0.15], 7) E [1.5, 2.5]
Moderate environments (e.g., BipedalWalker)::
Over E [0.04, 0.07]. 0 grad [0.008, 0.015), Oeure 0.53
fi E [..0,3.0] X E [0.1, 0.2). 17 E 3108,3.0]
Complex environments (e.g., Humanoid):
-ver E [0.06, 0.1], grod E [0.015, 0.025]. Oewry E [0.6, 1.0]
Kr [.55, 3.5), Ào E [0.15, 0.25]. 77 E [2.5, 4.0]
Overall, we found that MA-MPPI performs stably within +:25% variation range of parameters, demonstrating good robustness for practical applications. The most sensitive parameters are the influence radius multiplier K, and base temperature λo while the least sensitive are the decay factor Bdecay and transition sharpness 3.
K
Analysis of Generalization Limitations
While the Memory-Augmented Potential Field theory demonstrates excellent performance in convex control problems, it exhibits certain limitations feature generalization. Specifically, the system shows limited ability generalize when encountering topological features that are similar but not identical to previously memorized features.
K.1 Current Generalization Limitations
In the current implementation, the memory module represents each topological feature (such as local minima, low-gradient regions, etc.) as discrete points in state space with an influence radius defining their range of effect. This representation works well when dealing with exact matches highly similar features but faces the following limitations:
The current method uses a collection dis- crete points represent memory features, lacking abstract representation of structural similarities between features.
Disereteness of feature representation:
2. Distance-based similarity metrics: The system primarily relies on Euclidean distance to assess similarity between current states and memorized features, which may not be sufficiently precise high-dimensional state spaces.
3. Simplistic feature consolidation: Current feature merging rules are primarily based on spatial proximity, failing to fully capture semantic or functional similarities between features.
These limitations become particularly evident when the controller encounters structurally similar obstacles control challenges in different regions of the state space. For example, in the UAV obstacle avoidance task, the system might not immediately recognize that two U-shaped obstacles with different orientations present similar navigation challenges, requiring similar escape strategies.
K.2 Potential Improvement Directions
To enhance the system's generalization capabilities, we propose several potential improvement directions:
1. Hierarchical feature representation: Introduce hierarchical feature representation that ab- stracts low-level specific features into higher-level patterns, enabling the system to recognize structurally similar topological features.
32

---

2. Manifold- based similarity metrics: Develop similarity metrics based on the local manifold structure of the state space rather than relying solely on Euclidean distance. This could involve techniques from topological data analysis or spectral methods.
3. Feature embedding learning: Represent features as vectors in a low-dimensional em- bedding space, using supervised self-supervised learning ensure functionally similar features are proximate in the embedding space.
4. Transfer learning mechanisms: Design explicit transfer learning mechanisms that allow the system to adapt control strategies learned from one feature and apply them to similar features.
Preliminary experiments suggest that even simple feature embedding and similarity learning can improve generalization between similar features by 25-40%, particularly when task variations are moderate. This indicates significant room for improvement in MA-MPPI's generalization capabilities through appropriate representation learning and knowledge transfer mechanisms.
K.3 Empirical Evidence
To quantify the current generalization limitations, we conducted an experiment where the controller was trained specific obstacle configurations and then tested on variations of these configurations. Table summarizes the performance degradation as function of feature variation.
Table 15: Performance degradation with feature variations
Feature type
Performance relative to original feature (%)
20% variation 40% variation
60% variation
80% variation
Local minima location
92.7
75.3
58.4
42.1
Obstacle shape
87.5
68.2
51.7
38.4
Corridor width
94.1
79.6
62.8
45.3
Multi-feature scenarios
84.3
63.7
46.5
32.7
The results show that performance degrades significantly with increasing feature variation, particularly in multi-feature scenarios where interactions between features create more complex topological structures.
Initial experiments with prototype embedding- based generalization mechanism show promising results, with significant improvements in transfer performance between similar features. This suggests that integrating modern representation learning techniques with the memory augmentation framework could substantially address the current generalization limitations while maintaining theoretical guarantees of the base approach.
Future work will focus on developing formal theoretical extensions to the Memory-Augmente Potential Field theory that explicitly account for feature similarity and knowledge transfer, providing similar convergence and optimality guarantees for generalized features as the current framework provides for explicitly memorized features.
33

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly articulate the paper's main contributions: (1) the Memory-Ause Augmented Potential Field Theory framework that integrates historical experience into stochastic optimal control, (2) theoretical analysis showing non-convex escape properties and convergence guarantees, (3) implementation a
Justification:
MPPI controller, and (4) experimental validation in challenging non-convex environments.
These claims accurately reflect the content presented throughout the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Section (Discussion and Limitations, lines 281-290) explicitly discusses several limitations of the current approach, including restricted generalization between similar features, lack of sophisticated memory management for extended operations, and absence of multi-agent knowledge sharing. Appendix K provides detailed analysis of the generalization limitations, specifically. Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
34

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
The theoretical results in Section 3.3 (Theorems 3.1, 3.2, and 3.3) are presented with clear assumptions and statements. Complete formal proofs for all theorems are provided in Appendix C, with proof sketches in the main text providing intuition.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper provides detailed experimental protocols in Sections 5. and 6. 1,1, with comprehensive implementation details in Appendices H and I. These sections cover en- vironment specifications, algorithm configurations, hyperparameter settings, and evaluation metrics sufficient for reproducing the main results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [Yes]
Justification: The paper states in line 55 that "The code has been anonymized and available at hthhtps::/ //gi thub. com ContinuumCoder///// wwaeicooeer/AAPPP [AAPF__MPP The experiments use standard benchmark environments (OpenAl Gym and MuloCo), and detailed implementation instruc- tions are provided in Appendices E-I.
Guidelines:
35

---

The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
guwdes/ CodeeSomiisssseee sooooiiccci for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The experimental settings are thoroughly documented in Sections 5.1, 6,1, and Appendices H and I. These sections detail environment configurations, evaluation protocols, hyperparameter settings (including how they were chosen), implementation details, and baseline configurations. Section provides additional sensitivity analysis for key hyperparameters.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: All experimental results in Tables 1-4 include standard deviations across multiple runs. The paper explicitly states in line 185 that 30 independent runs with different random seeds were conducted for statistical robustness. Section I.I mentions that statistical significance was assessed using paired t-tests with Bonferroni correction.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
36

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Section 1.1.2 specifies the computational resources: "All experiments were conducted on a computing cluster with Intel Core i9- 12900K CPUs (3.20GHz) and NVIDIA RTX 3070 GPUs." Section H.5 provides detailed analysis of computational overhead and execution times for different components of the algorithm.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:p::://eerppps.cc/////////t ctc.heeeeeelscuiiiieeeeeiiee
Answer: [Yes]
Justification:
The research involves purely algorithmic and simulation-base experimentation
without human subjects, real-world deployment risks, or privacy concerns. The paper includes a thorough discussion of broader impacts in Appendix A, addressing potential benefits and harms in accordance with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Appendix A (lines 341-372) provides comprehensive discussion of both positive societal impacts (improved safety and reliability critical applications, energy efficiency, democratized access to advanced control) and negative impacts (workforce transitions, dual-use concerns, overreliance, accountability challenges). The section also suggests mitigation strategies.
37

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymenn of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications,
authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper introduces a control algorithm and does not release high-risk models datasets that have potential for misuse. The work focuses robotic control tasks in simulation environments rather than generating content or processing sensitive data.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The paper properly cites the original sources for all environments and baseline methods used in the experiments (OpenAl Gym, MuloCo, and various control algorithms) in the references section. These are standard benchmark environments in the research community, used appropriately for academie research purposes.
Guidelines:
The answer NA means that the paper does not existing assets.
38

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the datasets.paperssst paperswi thcode. cow /atasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged to reach out to the asset's ss creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces a new algorithm (MA-MPPI) which is thoroughly documented Sections 3 and 4, with implementation details in Appendices E-G. The code repository mentioned in line 55 contains documentation for using the implementation.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing or research with human subjects. All experiments are conducted in simulation environments with algorithmic agents.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for rescarch with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
39

---

Answer: [NA]
Justification: The paper does not involve human subjects research, so IRB approval was not required. All experiments were conducted in simulation environments.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
The research does not Large Language Models as components in its methodology. The Nemory-Augmente MPPI approach relies on control theory, dynamics models, and memory structures without incorporating LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy hattps://neur// hatprr/ppp..cccoooffeeeeeee///2 for what should or should not be described.
40


Reviews:
[{"review_id": "eZkcYLGFkX", "reviewer": "Reviewer_Erbx", "summary": "This paper introduces Memory-Augmented Potential Field theory, a novel mathematical framework aimed at enhancing stochastic optimal control by leveraging historical trajectory data. The method addresses the common issue of control algorithms becoming trapped in local minima within non-convex optimization landscapes. It identifies and memorizes three critical topological features—local minima, low-gradient areas, and high-curvature regions—to dynamically reshape the potential field landscape for improved exploration and convergence. The authors rigorously establish theoretical guarantees, including the ability to escape from local minima and asymptotic convergence to global optima. Additionally, the paper presents a practical implementation called Memory-Augmented Model Predictive Path Integral (MA-MPPI), demonstrating significant performance improvements across various complex tasks, including robotics, power system control, and UAV obstacle avoidance.", "questions": "- Could the authors clarify the fairness of the comparison with model-free RL methods (SAC, PPO), especially regarding interaction budget and network sizes?\n- Can the authors elaborate on how RL methods could achieve 80% of asymptotic performance within such limited interactions?\n- Given the limited literature review (only 20 citations), could the authors expand and discuss additional relevant works that directly compare or relate to their approach?", "limitations": "- Explicit memorization of topological features may struggle in rapidly changing or highly uncertain environments, necessitating continuous updates.\n- The method assumes perfect state information. Addressing uncertainties in state estimation (e.g., sensor inaccuracies in UAV positioning) could substantially strengthen real-world applicability.\n- Trapping can also arise from semantic features not easily captured by rule-based methods used here. Extending the approach to include learning-based methods for feature extraction could be an intriguing avenue for future research.\n- Clarifying how memory scales with an increasing number of memorized features through an ablation study on memory size would greatly enhance understanding of practical memory limitations (strongly recommended upon acceptance).", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 4, "originality": 3, "strengths_and_weaknesses": "[Strengths]\n- The proposed memory-based potential field utilizing historical observations is novel and well-motivated, providing a theoretically grounded approach to address the critical problem of local minima trapping.\n- The authors rigorously prove important theoretical properties, providing strong mathematical assurances for their proposed framework.\n- Extensive and convincing empirical evaluations demonstrate the method’s advantages over state-of-the-art methods in diverse tasks, encompassing both model-free and model-based approaches.\n- Comprehensive supplemental material thoroughly documents implementation details and hyperparameter sensitivity, aiding reproducibility and deeper understanding.\n\n[Weaknesses]\n- The comparison between model-based and model-free methods lacks sufficient clarity regarding experimental settings, even after consulting Appendix H.1.2. Specifically, it is unclear how model-free RL algorithms (e.g., SAC/PPO) were trained fairly, given that these methods typically require significantly more interactions (on the order of millions) and network capacity to achieve strong performance.\n- The interpretation of sample efficiency results remains ambiguous. The stated metric—\"environment interactions required to reach 80% of asymptotic performance\"—seems questionable, as typical RL methods generally require far more interactions (often tens of thousands for initial convergence alone).\n- (minor) The provided code is not immediately runnable or complete, containing several \"ToDos,\" suggesting it is not fully ready for public use. Given the code's prominent mention, ensuring its readiness would greatly benefit readers and the community.", "was_revised": true, "final_justification": "The authors have resolved most of my questions, and I would maintain the overall score as 5: Accept.", "rebuttal": "Thank you very much for taking the valuable time to review our manuscript and providing such insightful and constructive feedback. Your recognition gives us great encouragement, and the issues you raised are crucial for improving the quality of our paper. We believe that after making revisions based on your suggestions, the clarity, rigor, and impact of the manuscript will be significantly enhanced.\n\nBelow are our detailed responses and explanations to the questions and concerns you raised.\n\n**Regarding Weaknesses 1, 2, and Questions 1, 2**:\n\nWe fully understand your concerns about the fairness of comparisons with methods like SAC/PPO. We acknowledge that the exposition in the original manuscript was not sufficiently clear, leading to misunderstandings about experimental settings and metrics, and we apologize for this.\n\nTo ensure a fair comparison of learning efficiency, we benchmarked the entire learning process from scratch for all methods. Each method was allotted an identical budget of 2000 online interactions with comparable offline computation. This constraint is crucial for fairness, as sampling-based controllers like MPPI perform extensive internal computations (i.e., simulating numerous trajectories) for each online interaction, making their computational load comparable to the offline network training in RL methods. The reported \"Average cumulative rewards\" metric reflects this complete history, including the crucial early exploration phase, thus providing a comprehensive measure of overall learning efficiency and stability.\n\nFor the sample efficiency metric (N80\\%): Our original intention was to measure the number of interaction steps required for a small-capacity reinforcement learning model to achieve stable performance under our online, data-limited setting. However, we completely agree with your view and recognize that the N80\\% metric itself is not clear and intuitive enough, and can easily cause misunderstandings. Therefore, in the revised version, we will remove this metric and its related discussion, and adopt more direct approaches (analyzing learning curves) to demonstrate the sample efficiency and learning stability of different methods.\n\n**Regarding Question 3**:\n\nThank you for pointing out the limitations in the literature review. We will expand the related work section in the revised version and add more relevant references. We have conducted preliminary supplementary experiments to quantify the core differences between our method and other MPPI variants (such as DRPA-MPPI and LOG-MPPI).\n\nPreliminary results show that when repeatedly encountering the same \"U-shaped\" traps, our MA-MPPI benefits from the memory mechanism, with the average escape time for subsequent traps reduced by more than 70\\% compared to the first encounter. In contrast, DRPA-MPPI without memory shows almost no change in escape time when encountering the same traps subsequently. We will include the above comparative analysis in the related work section of the revised version and commit to providing complete experimental details and data for these comparisons in the final version of the paper.\n\n**Regarding Weakness 3**:\n\nWe apologize and, while currently restricted by the latest rebuttal policies from updating our repository, will organize and improve as soon as possible to provide a clearer, more user-friendly code repository that facilitates community reproduction and research.\n\n**Regarding \"Limitations\"**:\n\nYour insights about the limitations of our method are very profound and highly inspiring for our future research directions. We completely agree with these observations.\n\nRegarding real-world challenges such as dynamic environments, state uncertainty, and semantic features, we will explicitly discuss these important issues in Section 7 (Discussion and Limitations) and position them as key directions for future work. We plan to explore more efficient memory update strategies, integration schemes with state estimation algorithms, and the introduction of learning-driven feature extraction mechanisms to address these challenges.\n\nRegarding ablation studies on memory capacity, we completely agree that this is an important evaluation dimension. We have conducted preliminary ablation studies, and results in the Humanoid-v4 environment show that when memory capacity increases from 10 features to 30 features, there is some improvement in convergence speed; beyond 30 features, performance gains tend to plateau, indicating that our memory merging and forgetting mechanisms can effectively control the accumulation of redundant information.\n\nWe will provide complete details of this ablation study in the revised version to enhance understanding of the practical boundaries of our method.\n\n**Regarding Format Issues**:\n\nWe accept your suggestions. We will move key results, such as Table 2, from the appendix to the main text  Section 5.2, and we will redraw the figures, significantly increasing their size and label fonts, and adding clear legend explanations to ensure they are intuitive and easy to understand."}, {"review_id": "rePFtiZt6q", "reviewer": "Reviewer_tVZE", "summary": "The paper addresses the problem of avoiding local optima in stochastic optimal control by the integrating the use of memory to the controller. To do so, the standard value function in model predictive path integral controller is augmented with the memory component. Experiments were conducted extensively in different domains to show the effective of the approach.", "questions": "In the scope of the paper, would it be cases where experiences from the past can limit the performance and how to avoid it from happening? As minor: In line 119-122, the reference to Morse theory should be included in place.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "The paper is well-written and self-contained. The approach is technically sound. Looking at the experiences in the past to improve predictions or learning process is proved to be an useful mechanism in many cases. This work further designed a framework to integrate this into model predictive path integral controller and show the positive results for the case of dynamical system. Experiments in Section 5 and Section 6 covers both robotics control task and more complex engineering systems such as power system control and aerodynamics control (quadrotor model). The results show significantly higher optima escape rates.", "was_revised": true, "final_justification": "In the rebuttal phase, my questions in the review have been adequately addressed. For final justification, I maintain the overall score of 5: Accept.", "rebuttal": "Thank you very much for acknowledging our work and raising this critical question about how past experiences might limit performance. We completely agree that this is a core challenge for any memory-based system, and our framework systematically mitigates this risk through several key mechanisms. First, our approach does not simply memorize \"bad trajectories,\" but rather learns and stores the topological features of state space (such as the locations of local minima) that lead to optimization stagnation. This higher-level abstraction is far more robust than memorizing specific paths. Second, the framework incorporates dynamic memory management mechanisms to handle inaccurate or outdated memories. The strength of each memory feature is not fixed but dynamically adjusted based on how frequently it is encountered, and if a memory region is not revisited for an extended period, its strength gradually weakens through the decay factor and is eventually removed. This \"forgetting\" mechanism ensures that the system can adapt to environmental changes and clear those transient or incorrectly identified \"traps.\" Furthermore, our feature merging algorithm consolidates spatially neighboring similar features, which prevents overfitting to single negative experiences and instead forms a more generalized understanding of problematic regions. Finally, the influence of memory potential fields is strictly limited to the neighborhoods of identified features through adaptive weights, and in unknown state spaces, the controller still relies on the original objective function for exploration, ensuring that historical experiences do not overly constrain exploration in new environments. Regarding the minor issue you mentioned, we completely agree and will supplement the relevant sections in Lines 119-122 of the revised manuscript with citations to Morse theory. \n\nThank you again for your support and recognition of our work, as well as this constructive feedback, which helps us better articulate the robustness of our approach, and we hope it contributes new perspectives and theoretical insights to the field."}, {"review_id": "6ZEK2ee5LK", "reviewer": "Reviewer_BpFZ", "summary": "This paper focuses on the Potential Field Theory approach. While I am not an expert on this topic, I must say that the authors have made it quite readable and have provided a good introduction to the methodology. Departing from previous methods, the authors propose a memory-augmented potential field. Following the rigorous theoretical treatment, the authors present experiments that seem to support their claims and demonstrate the interesting features of their approach, analyzing memory’s impact on non-convex optimization topologically. Overall, I found the paper to be quite challenging yet also stimulating and inspiring.", "questions": "How does computational overhead scale for long deployments (e.g., >10k steps) as memory grows? Are there pruning heuristics?\n\nHow does the method handle dynamic environments? The framework is shown to be effective at building a memory of static non-convex landscapes. How would the MA-MPPI controller perform in an environment where the \"traps\" or obstacles change position over time?\n\nTheorem 3.1 guarantees escape from local minima by creating an \"outward push\" that overcomes the pull of the value function. But why will this push move the system into the correct safe region/direction? What if the memory consists of incorrect trajectories?\n\nIs there a principled way to set detection thresholds $\\theta_{grad}$ (Appendix B) for new tasks?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper is excellent on the presentation side in regards to technical ideas, whilst the contributions are novel/creative they are presented in such a way that understanding them was not difficult. The new formulation allows for an advanced dynamical adjustment of the decision strategies based on prior experience when exploring complex environments. Allowing to build a memory mechanisms with dynamical systems theory and stochastic optimal control.", "was_revised": true, "final_justification": "Thank you. I maintain my original score.", "rebuttal": "We are very grateful for these questions that precisely touch upon the core aspects of our work, including scalability, robustness, theoretical foundations, and practical applicability. To directly address your concerns about long-term deployment scalability, we specifically conducted a new supplementary experiment in a complex control task we designed with a high-dimensional state space, extending the deployment duration to over 10,000 control steps. The preliminary results: as the system explores more deeply, the size of the memory module does not grow indefinitely but tends to stabilize after reaching a certain number, and the total computational overhead correspondingly remains within bounded limits. This is attributed to our designed feature merging and dynamic forgetting mechanisms, which together constitute an effective implicit pruning strategy. To further test the framework's adaptability to dynamic environments, we also designed an additional UAV navigation task in revision where the obstacle layout undergoes dynamic reconfiguration during task execution. Our preliminary experiments confirm that MA-MPPI can successfully adapt to this change: memories of old obstacle positions rapidly decay due to no longer being triggered, while the system forms new memory features at new positions, quickly recovering efficient navigation after experiencing brief performance fluctuations. Regarding your profound insight about theoretical guarantees, specifically why \"pushing outward\" is safe, the key lies in understanding that its design purpose is not to provide a single \"correct global direction\" but to ensure that the system can escape the attraction basin of local minima. For local minima, our designed repulsive potential field is isotropic, with its function being \"escape\" rather than \"navigation,\" while directional guidance is only used for handling other feature types such as low-gradient regions. Therefore, even if the system occasionally memorizes inaccurate traps, such memories will be quickly corrected through the decay mechanism due to subsequent non-stagnation at those locations, thus ensuring system robustness. Finally, regarding parameter setting as a key practical issue, we have completed a comprehensive sensitivity analysis with preliminary conclusions showing that our algorithm performs stably within ±25\\% variation range of key detection thresholds, and we can extract a principled initial parameter setting guide based on the intrinsic complexity of tasks (such as state space dimensions). We believe these new experiments and detailed explanations can directly address your concerns, and we commit to providing complete details and data of these supplementary experiments in the revised paper to more solidly support the practicality and robustness of our framework."}, {"review_id": "ZQdA4IYVMb", "reviewer": "Reviewer_CQ9T", "summary": "This paper proposes an enhancement to the potential field algorithm by incorporating memory features(past experienced features) to address the non-convex landscape of the value function in stochastic optimal control, specifically within the framework of Model Predictive Path Integral control (MPPI). \n\nThe authors aim to achieve a seamless integration of memory-augmented features with existing MPPI-based value function frameworks. By storing information about previously encountered non-convex regions, the method enables the controller to recognize and compensate for similar situations when they recur, thereby improving robustness in complex environments.", "questions": "**1.** The method increases exploration in memorized regions by modulating the sampling temperature. Conventionally, exploration is often adjusted by modifying the sampling covariance[4, 5] . Why was temperature scaling (via the $\\lambda$ parameter) chosen instead? Does this approach offer practical or theoretical benefits over covariance-based methods?\n    \n**2.** The paper includes evaluations against multiple controllers, but the details of the cost function setup—particularly for non-RL baselines—are not fully explained. For instance, in Section 6.3 (UAV Obstacle Avoidance), how is obstacle avoidance encoded into the cost function for each controller?\n    \n**3.** MPPI is a widely used algorithm in control and robotics. Can the proposed memory-augmented potential field be integrated with alternative MPPI formulations such as constrained or smoothed MPPI frameworks [6, 7]? If so, is the proposed method always helpful for the other MPPI variants?\n    \n**4.** How does the proposed method compare with other recent MPPI-based approaches that also aim to address non-convexity, such as LOG-MPPI [1] or DRPA-MPPI [2]? A comparative study or discussion would clarify the strength.\n\n\n[1] Mohamed, I. S., Yin, K., & Liu, L. (2022). Autonomous navigation of agvs in unknown cluttered environments: log-mppi control strategy. _IEEE Robotics and Automation Letters_, _7_(4), 10240-10247.\n\n[2] Fuke, T., Endo, M., Honda, K., & Ishigami, G. (2025). DRPA-MPPI: Dynamic Repulsive Potential Augmented MPPI for Reactive Navigation in Unstructured Environments. _arXiv preprint arXiv:2503.20134_.\n\n[3] Fujimoto, Scott, et al. \"For sale: State-action representation learning for deep reinforcement learning.\" Advances in neural information processing systems 36 (2023): 61573-61624.\n\n[4] Xue, H., Pan, C., Yi, Z., Qu, G., & Shi, G. (2024). Full-Order Sampling-Based MPC for Torque-Level Locomotion Control via Diffusion-Style Annealing. _CoRR_.\n\n[5] Yi, Z., Pan, C., He, G., Qu, G., & Shi, G. (2024, June). CoVO-MPC: Theoretical analysis of sampling-based MPC and optimal covariance design. In _6th Annual Learning for Dynamics & Control Conference_ (pp. 1122-1135). PMLR.\n\n[6] Balci, I. M., Bakolas, E., Vlahov, B., & Theodorou, E. A. (2022, June). Constrained covariance steering based tube-mppi. In _2022 American Control Conference (ACC)_ (pp. 4197-4202). IEEE.\n\n[7] Kim, T., Park, G., Kwak, K., Bae, J., & Lee, W. (2022). Smooth model predictive path integral control without smoothing. _IEEE Robotics and Automation Letters_, _7_(4), 10406-10413.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n**1**. The paper presents a theoretically grounded extension to MPPI by introducing a memory-augmented potential field algorithm, aimed at addressing non-convexities in the value function.\n    \n**2**. authors provide formal analysis showing improvements in non-convex escape capability, convergence rate, and efficiency over standard MPPI.\n    \n**3**. The proposed method is extensively validated through a variety of experiments, demonstrating its practical advantages in complex control tasks.\n    \n\n### Weaknesses\n\n**1.** The paper suffers from organizational and clarity issues. While the integration of the memory mechanism is technically interesting, the exposition lacks conciseness, making it difficult to follow in several places. In particular, some figures are difficult to interpret due to small font sizes and the lack of detailed explanations for individual labels and components.\n    \n**2.** While experimental results show promising performance, the paper lacks comparative analysis with other recent methods that also tackle non-convexity in control, particularly those using memory-augmented or other sampling methods[1, 2].\n\n**3.** A significant concern lies in the reported performance of baseline algorithms, particularly SAC and DDPG, on standard MuJoCo benchmarks such as HalfCheetah-v4 and Humanoid-v4. The performance values reported by the authors are substantially lower than those widely accepted in prior literature and established implementations.\n\nFor example, in [3], the asymptotic performance of SAC exceeds 15,000 on HalfCheetah-v4 and 6,500 on Humanoid-v4. In contrast, the authors report only 5,842 and 4,125, respectively. Similar discrepancies are consistently observed when comparing with well-maintained open-source implementations such as CleanRL.\n\nSince SAC and DDPG are widely used and well-established baseline algorithms, deviations of this magnitude require careful validation and justification. Without such support, **the credibility of the experimental results is seriously undermined.**\n\n------\n[1] Mohamed, I. S., Yin, K., & Liu, L. (2022). Autonomous navigation of agvs in unknown cluttered environments: log-mppi control strategy. _IEEE Robotics and Automation Letters_, _7_(4), 10240-10247.\n\n[2] Fuke, T., Endo, M., Honda, K., & Ishigami, G. (2025). DRPA-MPPI: Dynamic Repulsive Potential Augmented MPPI for Reactive Navigation in Unstructured Environments. _arXiv preprint arXiv:2503.20134_.\n\n[3] Fujimoto, Scott, et al. \"For sale: State-action representation learning for deep reinforcement learning.\" Advances in neural information processing systems 36 (2023): 61573-61624.", "was_revised": true, "final_justification": "I reviewed every discussion and raised the score 4.", "rebuttal": "We sincerely appreciate the valuable time and profound insights you have dedicated to reviewing our manuscript. Your comments are not only insightful but also highly constructive. We believe that after revising according to your suggestions, the manuscript's clarity, rigor, and impact will be significantly enhanced. Below are our responses and clarifications to the weaknesses and issues you raised regarding the paper.\n\n**Regarding Weakness 1**:\n\nWe greatly appreciate your valuable feedback on the organizational structure and clarity of the paper, and we acknowledge that clear exposition is crucial. To address the issues you identified, we will add more intuitive explanations for key methodological sections in the revised version to help readers better understand the core ideas behind our framework. Additionally, specifically regarding the interpretability of figures, we have increased the font sizes in the figures and provided more detailed caption explanations for each label and component.\n\n**Regarding Weakness 2 and Question 4**:\n\nThe core distinction between MA-MPPI and other MPPI variants lies in its online learning mechanism based on persistent memory. DRPA-MPPI is an event-triggered reactive approach that temporarily switches cost functions by analyzing current predicted trajectories to escape traps. LOG-MPPI is a sampling-based exploratory method that fundamentally improves exploration efficiency by using normal log-normal mixture distributions to probabilistically circumvent local minima. Unlike these two methods, MA-MPPI can learn from historical experience, dynamically constructing and memorizing topological features of the state space (such as local minima, low-gradient regions, etc.). This memory mechanism enables it to proactively avoid known trap regions in subsequent navigation, demonstrating efficiency far superior to reactive or purely exploratory methods in environments containing repetitive challenges. To preliminarily quantify these differences, we conducted an initial test evaluating escape efficiency in an environment containing multiple identical \"U-shaped\" traps. We focused on two metrics: \"first trap escape time\" and \"subsequent trap average escape time.\" Preliminary results show that for initially encountered traps, MA-MPPI performs comparably to DRPA-MPPI, as both can effectively detect and escape. However, for subsequently encountered identical traps, MA-MPPI's average escape time was reduced by over 70\\% because it utilized memory to proactively avoid those regions. In contrast, DRPA-MPPI, lacking memory, showed virtually no change in subsequent escape times compared to the first encounter. While LOG-MPPI could also succeed, its escape times were longer with greater variance in all cases, reflecting the nature of its probabilistic exploration. This result clearly demonstrates MA-MPPI's core advantage: truly learning from experience to avoid repeating mistakes, rather than merely solving single-instance problems. We will provide complete experimental comparisons, including these metrics, in the revised version. We will add this comparison to the \"Related Work\" section or a newly added \"Discussion\" section of the revised paper, with complete quantitative analysis in future experiments. \n\n**Regarding Weakness 3**:\n\nPlease allow us to clarify: Considering that fields such as robotics need to interact directly with the physical world, each interaction is costly. Our goal is to evaluate the algorithm's adaptability and learning stability under a comparable total computational budget (combining online interactions and offline simulations), rather than the ultimate performance achievable after massive training. The \"Average cumulative rewards\" we report for RL baselines represents their average performance across this entire budgeted learning history, which is affected by lengthy exploration phases and is completely consistent with implementations achieving peak performance of 10000+. Additionally, our scores are the average results of 30 independent random seed experiments, which naturally include runs that failed to achieve single-trial optimal performance due to initialization and exploration randomness. This is fundamentally different in nature from peak performance that might be reported in other literature after extensive hyperparameter tuning and selection. The scores in the literature [3] you cited, which we verified are based on Gym 0.25, are likely not the v4 version. There are substantial differences in dynamics and reward functions between different Mujoco versions, making cross-version scores not directly comparable. The revised version will have clearer, more rigorous expressions and provide more details.\n\n**Regarding Question 1**:\n\nWe chose to implement exploration strategies by adjusting sampling temperature ($\\lambda$) because this directly relates to MPPI's path integral control theory, and our framework actually uses temperature scaling as a principled way to achieve covariance scaling, rather than making the two mutually exclusive. This approach forms a two-layer adaptive framework: the first layer, where memory potential fields guide exploration direction by reshaping the value function; the second layer, where elevated temperature ($\\lambda$) correspondingly increases sampling covariance ($\\Sigma$) to strengthen random exploration intensity in these critical regions. This synergy between guided exploration and random exploration can more efficiently discover escape paths created by the memory module, while maintaining high adaptability with MPPI's theoretical foundation.\n\n**Regarding Question 2**:\n\nThank you for raising this important question about cost function settings. We acknowledge that this technical detail is crucial for establishing experimental validity, and a detailed explanation of cost function settings indeed needs to be sufficient in the article. To ensure fair comparison, we designed structurally identical cost functions for all controllers, including four components: goal-reaching cost, obstacle avoidance penalty, control effort regularization, and trajectory smoothness term. The obstacle avoidance component specifically uses distance-based penalty functions, applying quadratic penalties when the drone approaches within the safety radius (0.5 meters) of any obstacle, with penalty strength inversely proportional to the square of distance. Additionally, we impose large terminal penalties (1000) for actual collisions, ensuring all methods prioritize safety. While maintaining the same underlying cost structure, we adapted representations for each controller type. For MPPI-based methods (including our MA-MPPI), we directly use the cost function for trajectory evaluation. For RL methods (SAC, PPO, DDPG), we convert costs to negative rewards with identical weights. Traditional MPC methods implement obstacle constraints through barrier functions, while iLQR uses quadratic approximations of barrier costs. These adaptations maintain optimization objective consistency while respecting each method's mathematical framework. We determined cost function weights through grid search optimization on standard MPPI (goal-reaching $\\lambda_1=1.0$, obstacle avoidance $\\lambda_2=5.0$, control effort $\\lambda_3=0.1$, smoothness $\\lambda_4=0.05$), then applied these same weights to all methods to eliminate tuning bias. Sensitivity analysis confirmed MA-MPPI maintains performance advantages within ±25\\% weight variation ranges, demonstrating our results' robustness to cost function parameterization. Our revised version expands Section 6.1 to include detailed cost function specifications. Additionally, we add an appendix in the revised version detailing technical specifics of UAV cost function settings, including exact mathematical formulas used for each baseline controller and method-specific wrapper implementations.\n\n**Regarding Question 3**:\n\nYes, it can. Our method is technically compatible and non-interfering with other MPPI variants because the memory-augmented potential field theory can work synergistically with different constraint handling and smoothness optimization frameworks. Our method primarily acts on the optimization objective: reshaping the value function through memory to generate a nominal trajectory that proactively avoids known traps and is globally more optimal. This allows underlying MPPI variants to focus on solving execution-level challenges. For example, Constrained MPPI can ensure trajectories strictly satisfy safety and physical constraints on this basis, while Smooth MPPI can focus on generating smooth control commands without high-frequency jitter. We conducted supplementary validation experiments combining MA-MPPI's memory mechanism with constrained MPPI frameworks. In 2D navigation tasks containing narrow passages, Combined MA-Constrained-MPPI improved success rates by approximately 15\\% compared to Constrained MPPI alone in repetitive tasks, while maintaining the same constraint violation rate (<2\\%). For smoothness integration, we observed that combining the memory-augmented potential field technique with Smooth MPPI reduced local minimum escape time by approximately 23\\% while maintaining trajectory smoothness metrics. We will include more integration analysis of the memory-augmented potential field method with major MPPI variants in the revised version, particularly combinations with constraint and smoothness frameworks."}]

Author rebuttal:

Meta-review: This paper introduces the theory of Memory-Augmented Potential Fields, a novel mathematical framework designed to enhance stochastic optimal control by leveraging historical trajectory data. Control algorithms are well known to often converge to local minima when applied to problems with non-convex optimization landscapes. To address this challenge, specifically within the framework of Model Predictive Path Integral control (MPPI), the proposed approach identifies and memorizes key topological features to dynamically reshape the potential field landscape and facilitate exploration and convergence. The authors established key theoretical guarantees, showing that the method achieves asymptotic convergence to global optima and can escape local minima. The performance of the proposed technique was evaluated across various tasks, including robotics, power system control, and UAV obstacle avoidance.

All reviewers agreed that the ideas explored in this work are novel, well-motivated, and clearly presented. They noted that the mathematical framework provides a principled approach to addressing convergence issues caused by local minima. From a technical perspective, reviewers found the proposed framework sound and supported by strong mathematical guarantees. Finally, all reviewers praised the well-designed and extensive empirical evaluations, which demonstrate the method's clear advantages compared to relevant competitors across a diverse suite of challenging problems.

The discussion phase between reviewers and authors was constructive. Reviewers raised a few concerns, most of which were relatively minor or were resolved after exchanges with the authors. For example, one reviewer expressed concern about the perceived lack of clarity in the comparison between model-based and model-free approaches and noted that some of the sample efficiency analyses were difficult to interpret. The authors provided a thorough and detailed response, which the reviewer found satisfactory, acknowledging that their main concerns had been adequately addressed. Another reviewer questioned whether the proposed method would scale to long-horizon problems, given that the computational overhead could, in theory, become intractable. In response, the authors conducted an additional experiment involving a task requiring 10,000 control steps. After this exchange, the reviewer confirmed that their main questions had been fully addressed and reiterated their strong positive view of the paper’s quality. Finally, another reviewer—the one who had initially raised the most pressing concerns—noted that the paper suffered from organizational and clarity issues and that a more thorough experimental setup was needed to fully support some of its central claims. The authors provided an extensive and detailed response addressing these points, which led the reviewer to significantly improve their recommendation.

In summary, all reviewers valued this paper’s creative and novel approach to addressing a challenging problem in stochastic optimal control. There was broad consensus regarding the quality of this work and the significance of its contributions, with reviewers noting that the broader NeurIPS community is likely to benefit from the insights introduced in this paper. The reviewers encouraged the authors to further improve the manuscript by addressing the points, questions, and concerns raised in the reviews and during the discussion phase.
Decision: Accept (poster)