SlowGuess/NeurIPS_2023_OCR huggingface dataset:
error reduction, rather than simply encouraging a refinement of areas with a high existing error regardless of the resulting mesh improvement. Further, incorporating a novel area scaling term encourages the policy to focus on smaller elements with a high potential reduction in error rather than larger elements with a low average error reduction, up to some threshold depending on the element penalty  $\alpha$ . We find that the combination of a local formulation and the area scaling term allows for a simpler credit assignment for the RL agents, as it ensures that each agent gets rewarded for its own actions and that rewards of elements of different sizes are on the same scale. Similarly, the area scaling term of the reward effectively cancels out the area of the integration points in Equation 2, causing the policies optimized on this reward to implicitly minimize the maximum remaining error of the mesh, while also making sure that the mean error stays sufficiently low. We compare this to directly minimizing the maximum error in Appendix C.2.
Since the effects of mesh refinement can be non-local for elliptical PDEs, we optimize the average of the local and global returns, i.e.,
$$
J _ {i} ^ {t ^ {\prime}} = \frac {1}{2} J _ {i} ^ {t} + \frac {1}{2} J ^ {t}, \tag {4}
$$
where  $J_{i}^{t}$  is the return of agent  $i$  at step  $t$  as shown in Equation 1, and  $J^{t}$  is the global return calculated using the average reward  $r = \frac{1}{N}\sum_{j}\mathbf{r}_{j}$ . In multi-quantity systems of equations, it is important for the mesh to be suitable for all the quantities of interest. For this, we calculate individual errors  $\mathrm{err}^d (\Omega_i^t)$  for each solution dimension and then use a norm or a convex sum of these as the overall error, depending on the application.
Setup. All learned methods are trained on 100 PDEs and their corresponding initial and reference meshes  $\Omega^0$ ,  $\Omega^*$  to limit the number of required reference meshes during training. We experiment with 10 different target mesh resolutions per method to produce a wide range of solutions, as detailed in Appendix F.3. We repeat each experiment for 10 random seeds and always report the average performance on 100 randomly sampled but fixed evaluation PDEs. These PDEs are disjoint from the training PDEs, and both sets of PDEs consist of randomly sampled domains as well as boundary and initial conditions as detailed below. Details on the setup and the computational budget for our experiments are provided in Appendix C.1. The reference mesh  $\Omega^*$  is created by uniformly refining the initial mesh 6 times. An environment episode consists of drawing one of the 100 training PDE without replacement, and iteratively refining the coarse initial mesh  $\Omega^0$  a total of  $T = 6$  times unless mentioned otherwise. Since the maximum number of elements scales exponentially with the refinement depth, we additionally evaluate a simpler task setup with  $T = 4$  refinements to roughly replicate the task complexity of existing work [28, 63, 29]. We experiment with Deep Q-Network (DQN) [86, 87] as an off-policy and Proximal Policy Optimization (PPO) [88] with discrete actions as an on-policy RL algorithm for all RL-based methods.
We evaluate mesh quality by calculating the squared error at each point in the high-resolution reference  $\Omega^{*}$ , i.e., as  $\sum_{\Omega_m^* \in \Omega^*} \mathrm{Area}(\Omega_m^*) \left(u_{\Omega^*}(p_{\Omega_m^*}) - u_{\Omega^t}(p_{\Omega_m^*})\right)^2$ , and normalize the resulting value by that of the initial mesh for comparability across PDEs. This metric captures both the maximum localized errors by punishing outliers, and the overall error across the domain. We evaluate both the mean error and an approximation of the maximum error over the mesh as additional metrics in Appendix D.6. Appendix F.1 lists all further algorithm and neural network hyperparameters.
Graph Features. The features  $\mathbf{X}_v$  of each node  $v\in \mathcal{V}$  consist of the environment timestep, the element area, the distance to the closest boundary, and the mean and standard deviation of the solution on the element's vertices. Edge features  $\mathbf{X}_e$  are defined as Euclidean distances between element midpoints. We omit absolute positions to ensure that the observations are equivariant under the Euclidean group [38, 37] to utilize the underlying symmetry of the task. We use additional task-dependent node features for some considered systems of equations, as described in Appendix B.
Systems of Equations We experiment on various 2D elliptical PDEs, namely the Laplace equation, the Poisson equation, a Stokes flow task, a linear elasticity example, and a non-stationary heat diffusion equation. The domains are L-shapes, rectangles with a square hole or multiple rhomboid holes, and convex polygons. Figure 2 shows exemplary ASMR refinements on all tasks and briefly explains the challenge associated with each task. The PDEs and the FEM are implemented using scikit-fem [89], and we use conforming triangular meshes and linear elements unless mentioned otherwise. The code provides OpenAI gym [40] environments for all tasks. We define the systems of equations and their specific features in Appendix B.
Baselines. We adapt several recent RL methods [28, 29, 30] that were originally designed for non-stationary AMR as baselines for our application focusing on stationary refinements. We use our error estimates as the basis of all reward calculations for comparability but otherwise calculate the rewards as described in the respective papers. Single Agent [28] predicts a categorical action over the mesh to mark the next element for refinement. Sweep [30] trains a single-agent policy by randomly sampling an element on the mesh and deciding its refinement based on local features and a global resource budget. During inference, each timestep consists of a sweep over the full mesh that may mark each element. Finally,  $VDGN$  [29] estimates a global Q-function as the sum of agent-wise local Q-functions. As the PPO version of  $VDGN$  has no Q-Function, we decompose the value function as the sum of value functions of the individual elements, yielding a  $VDGN$ -like baseline in the case of the PPO version. We use an MPN policy for Single Agent and  $VDGN$ , while Sweep utilizes a simple MLP. Hyperparameters and further details are provided in Appendix F.2
We also compare to a traditional error-based Oracle Error Heuristic [90, 91, 30]. Given a refinement threshold  $\theta$ , the Oracle Error Heuristic iteratively refines all elements  $\Omega_i^t$  for which  $\mathrm{err}(\Omega_i^t) > \theta \cdot \max_j \mathrm{err}(\Omega_j^t)$ . As we are also interested in the reduction of the maximum error, we analogously define the Maximum Oracle Error Heuristic, which uses the maximum error per element  $\max_{\Omega_m^* \subseteq \Omega_i^t} |u_{\Omega^*}(p_{\Omega_m^*}) - u_{\Omega^t}(p_{\Omega_m^*})|$  as a surrogate error estimate. Note that these baselines require the fine-grained reference mesh  $\Omega^*$ , which is expensive to compute and thus usually unavailable during inference. As a substitute, we consider the commonly used ZZ Error, which uses the superconvergent patch recovery process to estimate an error per mesh element [21]. Similar to the Oracle Error Heuristic, these estimates are combined with a refinement threshold  $\theta$  to iteratively refine the mesh. The ZZ Error generally produces smooth error estimates as the recovery process requires averaging over neighboring mesh elements, which can in some cases lead to more coherent refinements when compared to the Oracle Error Heuristic. The heuristics act on local element information and greedily refine elements with a high error rather than elements for which a refinement would lead to a high reduction in error. As such, they may select sub-optimal refinements for globally propagating errors, which is a well-known issue for elliptic PDEs [92, 30]. In contrast, RL methods learn to directly maximize the decrease in error, allowing them to find long-term strategies that also take the local receptive fields of the individual agents into account.
Additional Experiments. We conduct a series of ablation experiments to determine which parts of ASMR make it uniquely effective. We look at both the area scaling and the spatial decomposition of the reward in Equation 3 and ablate different node features and the number of training PDEs that are used. Additionally, we consider an alternate reward formulation that uses the maximum error per element instead of its average as detailed in Appendix C.2. Due to their importance for practical applications, we further experiment with both generalization to unseen and larger domains and the improvements in runtime for ASMR compared to a uniform refinement.
Quantitative Results. We visualize the mesh quality quantitatively with a Pareto plot of the number of elements and the remaining error. We plot one point per trained policy, which represents the interquartile mean [93] of this policy when evaluated on 100 evaluation environments. We further
Figure 4: Pareto plot of normalized squared errors and number of final mesh elements for Laplace's Equation. (Left) For only 4 refinement steps, all learned methods perform well and significantly improve upon a uniform refinement. (Right) Scaling to 6 refinement steps, the learned baselines become less stable and in some cases fail to provide refinements that are better than uniform. In contrast, ASMR consistently provides high-quality refinements and is on par with or better than the heuristics in both cases.
provide a log-log quadratic regression over the aggregated results of each method as a general trend-line. To enhance visibility and focus on typical behavior, we exclude sporadic outliers from the baseline methods that produce degenerate meshes with an excessively high number of elements. For all learned methods, we experiment with both PPO and DQN as the RL backbone on the Poisson equation in Appendix D.1. All learned methods, including ASMR yield better results with PPO, indicating that an on-policy algorithm is favorable when dealing with action and observation spaces of varying size. Similarly, we compare the Graph Attention Network (GAT)-like [94] architecture proposed by  $VDGN$  [29] to MPNs in Appendix D.2, finding that ASMR works well for both architectures, while  $VDGN$  performs better with MPNs. We consequently use PPO and MPNs in all other experiments. Appendix D.3 compares the ZZ Error estimator for different initial refinement levels. As the results show that a sufficiently fine initial mesh is important, we start each refinement procedure for the ZZ Error Heuristic with two uniform refinements. We note that this initial tuning prevents coarse refinements and is not needed for our method.
Using these results, Figure 4 compares the different approaches on Laplace's equation. The left side of Figure 4 shows that all methods work in a simple setup on par with experiments from previous work. Here, 4 refinement steps are used for all methods except for Single Agent, which instead refines 4 times fewer elements. On the right side, scaling to 6 refinement steps and significantly more elements, only ASMR effectively handles larger instances while learned methods falter. Notably, ASMR also outperforms the Oracle, Maximum Oracle, and ZZ Error Heuristics. These results demonstrate the effectiveness of our Swarm RL framework for learning non-greedy refinement strategies for static meshes. Specifically, ASMR refines elements with a high potential for error reduction over the heuristics' strategy of targeting elements with high error. Figure 5 provides results on the remaining tasks. Appendix D.6 presents additional results using a mean error and a smooth version of a maximum error metric. ASMR clearly outperforms the learned baselines on all tasks and is generally competitive with or better than the Heuristics. Both heuristics improve over the RL methods on the Stokes flow task, likely because the task requires high precision for both the inlet and on inner boundaries near regions of high flow velocity.
Qualitative Results. Figure 2 shows refinements of ASMR on randomly sampled systems of equations for all considered tasks. The refinement strategy adapts to the given task, providing an efficient trade-off between simulation accuracy and the number of elements used. Figure 3 visualizes the refinements of ASMR on a randomly sampled heat diffusion problem. ASMR refines based on the element penalty  $\alpha$ , yet always focuses on the heat source and its path. Appendix G.1 provides additional ASMR visualizations for all tasks, and Appendix G.2 visualizes all methods on Poisson's equation to showcase common refinement behaviors. Appendix G.3 presents the iterative marking procedure of our approach on an exemplary Poisson problem.
Elements  $(\times 10^{3})$
Heat Diffusion
Elements  $(\times 10^{3})$
Figure 5: Pareto plot of normalized squared errors and number of final mesh elements across different tasks. All methods generally work well for relatively small instances, but Single Agent, VDGN-like and Sweep break down for larger meshes. Our method uniquely scales to meshes to thousands of elements and consistently outperforms these learned baselines on all tasks and while performing on par with or better than the Oracle, Maximum Oracle and ZZ Error Heuristics in most cases.
Ablations. The reward proposed in Equation 3 combines an area scaling per element with a spatial allocation of the decrease in error to the individual mesh elements. Figure 6 investigates these decisions. We find that combining both features is uniquely responsible for the effectiveness of our method, suggesting that the spatial reward's limited expressiveness for small elements is compensated by area scaling. However, the area scaling can only be leveraged if it is allocated to individual mesh elements, as it may introduce excessive reward noise on the full mesh. The maximum reward variant of Appendix C.2 explicitly minimizes the maximum error of the mesh, resulting in refinements of similar quality than those created by ASMR using the reward in Equation 3. We thus use the latter as it simplifies the error estimate in the reward function and better aligns with existing work.
We evaluate the effect of different parameters for the target mesh resolution in Appendix D.4, finding that ASMR provides meshes with considerably more consistent numbers of elements for a given target resolution than the other learned methods. Additional ablations in Appendix D.5 show that 100 training PDEs are sufficient and that adding absolute positions in the node features is detrimental, while providing solution information and load function evaluations improves performance.
Generalization Capabilities and Runtime Experiments. Table 1 compares the wall-clock time of ASMR trained on a variant of the Poisson task with that of the reference  $\Omega^{*}$ , showing that our method provides a speedup of more than factor 100 compared to computing a uniform mesh for large domains. The evaluation uses load functions with 16 Gaussian modes and spiral-shaped of varying sizes for the same average initial element size. Appendix E.1 provides details for the training environments and the spiral-shaped evaluation domain, as well as results on larger domains and the associated improvement in runtime. These results include an ASMR visualization of a refinement of a  $20 \times 20$  spiral domain with more than 50 000 elements. Appendix E.2 additionally shows the exceptional generalization capabilities of ASMR across various domains and load functions for Poisson's equation on  $1 \times 1$  domains. Appendix E.3 presents further runtime comparisons for all tasks and shows that ASMR provides a task-dependent speedup of factor 2 to 30 over a uniform refinement.
Figure 6: Pareto plot of normalized squared errors and number of final mesh elements for the Linear Elasticity task for different reward ablations. ASMR benefits from the area scaling term of Equation 3 and a spatial reward formulation. The maximum reward of Appendix C.2 performs similar to that of Equation 3.
Domain	Elements	Time[s]	Speedup[×]
2 × 2	4208	0.19	14.6
3 × 3	4185	0.23	29.6
4 × 4	4391	0.28	47.1
5 × 5	5120	0.35	67.3
6 × 6	5462	0.41	90.1
7 × 7	6515	0.51	114.8
8 × 8	7506	0.60	131.4
Table 1: Elements and speedup versus uniform refinement for achieving a normalized squared error of 0.001 for different domain sizes on Poisson's Equation. The elements required for the error threshold grow slower than the domain size, indicating fewer areas of significant error in larger domains. Thus, ASMR offers increasing speedups as the size of the domain increases.
The generalization capabilities in combination with the fast runtime of our method offer substantial advantages in practical engineering applications. A policy can be trained on small, cost-effective environments and then deployed on much larger and dynamically changing setups during inference. These generalization traits arise from the MPN architecture and the utilized observation graphs, which both lead to refinement strategies based on local element neighborhoods rather than global meshes.
We present a novel Adaptive Mesh Refinement method that uses Swarm Reinforcement Learning to iteratively refine meshes for efficient solutions of Partial Differential Equations. Our approach, Adaptive Swarm Mesh Refinement (ASMR), treats each mesh element as an agent and trains all agents under a shared policy using Graph Neural Networks and a novel per-agent reward formulation. ASMR gracefully scales to meshes with thousands of elements without requiring an error estimate during inference. In our experiments focused on static meshes, ASMR demonstrates strong performance in handling complex refinements. The method significantly outperforms both existing Reinforcement Learning-based approaches and traditional refinement strategies, achieving a mesh quality comparable to expensive oracle-based error heuristics. Once trained, the ASMR policy generalizes well to different forcing functions and significantly larger problem domains. In terms of runtime, our method outperforms uniform refinements by up to 30 times on domains similar in scale to the training set, and by over 100 times in larger evaluation setups.
Broader Impact Our proposed Adaptive Mesh Refinement technique can positively impact various fields relying on computational modeling and simulation. By reducing simulation times while maintaining high precision, this technology enables researchers to explore a wider range of scenarios. However, like any powerful tool, there are potential negative impacts, such as the development of advanced weapon models or exploitation of resources.
Limitations and Future Work Our approach solves the partial differential equation after each refinement step, which requires a considerable amount of computation time. In future work, we will explore using Swarm RL for refinement strategies from the raw geometry and boundary conditions to further speed up our approach. We currently use relatively simple message passing networks for our policy, and want to optimize the network architecture to include, e.g., long-range message passing. Lastly, this work only considers 2D problems, static meshes with triangular elements, and comparatively simple domains. Here, we want to extend and modify our approach to quadrilateral meshes, time-dependent refinement and coarsening operations, and 3-dimensional domains.
NF was supported by the BMBF project Davis (Datengetriebene Vernetzung für die ingenieurtechnische Simulation). This work is also part of the DFG AI Research Unit 5339 regarding the combination of physics-based simulation with AI-based methodologies for the fast maturation of manufacturing processes. The financial support by German Research Foundation (DFG, Deutsche Forschungsgemeinschaft) is gratefully acknowledged. The authors acknowledge support by the state of Baden-Württemberg through bwHPC, as well as the HoreKa supercomputer funded by the Ministry of Science, Research and the Arts Baden-Württemberg and by the German Federal Ministry of Education and Research.
[1] Susanne C Brenner and L Ridgway Scott. The mathematical theory of finite element methods, volume 3. Springer, 2008.
[2] Junuthula Narasimha Reddy and David K Gartling. The finite element method in heat transfer and fluid dynamics. CRC press, 2010.
[3] Junuthula Narasimha Reddy. Introduction to the finite element method. McGraw-Hill Education, 2019.
[4] Robert Anderson, Julian Andrej, Andrew Barker, Jamie Bramwell, Jean-Sylvain Camier, Jakub Cerveny, Veselin Dobrev, Yohann Dudouit, Aaron Fisher, Tzanio Kolev, et al. Mfem: A modular finite element methods library. Computers & Mathematics with Applications, 81:42-74, 2021.
[5] Tomasz Plewa, Timur Linde, V Gregory Weirs, et al. Adaptive mesh refinement-theory and applications. Springer, 2005.
[6] Weizhang Huang and Robert D Russell. Adaptive moving mesh methods, volume 174. Springer Science & Business Media, 2010.
[7] Krzysztof J Fidkowski and David L Darmofal. Review of output-based error estimation and mesh adaptation in computational fluid dynamics. AIAA journal, 49(4):673-694, 2011.
[8] Marsha J Berger and Phillip Colella. Local adaptive mesh refinement for shock hydrodynamics. Journal of computational Physics, 82(1):64-84, 1989.
[9] Timothy J Baker. Mesh adaptation strategies for problems in fluid dynamics. Finite Elements in Analysis and Design, 25(3-4):243-273, 1997.
[10] Raunak Borker, Daniel Huang, Sebastian Grimberg, Charbel Farhat, Philip Avery, and Jason Rabinovitch. Mesh adaptation framework for embedded boundary methods for computational fluid dynamics and fluid-structure interaction. International Journal for Numerical Methods in Fluids, 90(8):389–424, 2019.
[11] Zheyan Zhang, Yongxing Wang, Peter K Jimack, and He Wang. Meshingnet: A new mesh generation method based on deep learning. In Computational Science-ICCS 2020: 20th International Conference, Amsterdam, The Netherlands, June 3-5, 2020, Proceedings, Part III 20, pages 186-198. Springer, 2020.
[12] Joseph Gregory Wallwork. Mesh adaptation and adjoint methods for finite element coastal ocean modelling. PhD thesis, Imperial College London, 2021.
[13] Joseph Gregory Wallwork, Jingyi Lu, Mingrui Zhang, and Matthew D Piggott. E2n: Error estimation networks for goal-oriented mesh adaptation. arXiv preprint arXiv:2207.11233, 2022.
[14] M Ortiz and JJ Quigley Iv. Adaptive mesh refinement in strain localization problems. Computer Methods in Applied Mechanics and Engineering, 90(1-3):781-804, 1991.
[15] Nikolas Provatas, Nigel Goldenfeld, and Jonathan Dantzig. Efficient computation of dendritic microstructures using adaptive mesh refinement. Physical Review Letters, 80(15):3308, 1998.
[16] Erwin Stein. Adaptive finite elements in linear and nonlinear solid and structural mechanics, volume 416. Springer Science & Business Media, 2007.