Title: SpiderSolver: A Geometry-Aware Transformer for Solving PDEs on Complex Geometries
Venue: NeurIPS 2025
Accepted: yes
Keywords: Machine learning in PDE; Transformer architecture; PDE on complex geometry; Tokenization;
Abstract: Transformers have demonstrated effectiveness in solving partial differential equations (PDEs). However, extending them to solve PDEs on complex geometries remains a challenge. In this work, we propose SpiderSolver, a geometry-aware transformer that introduces spiderweb tokenization for handling complex domain geometry and irregularly discretized points. Our method partitions the irregular spatial domain into spiderweb-like patches, guided by the domain boundary geometry. SpiderSolver leverages a coarse-grained attention mechanism to capture global interactions across spiderweb tokens and a fine-grained attention mechanism to refine feature interactions between the domain boundary and its neighboring interior points. We evaluate SpiderSolver on PDEs with diverse domain geometries across seven datasets, including cars, airfoils, blood flow in the human thoracic aorta, as well as canonical cases governed by the Navier-Stokes, Darcy flow, elasticity, and plasticity equations. Experimental results demonstrate that SpiderSolver consistently achieves state-of-the-art performance across different datasets and metrics, with better generalization ability in the OOD setting. The code is available at https://github.com/Kai-Qi/SpiderSolver.


Peer reviews:
comments: SUMMARY:
The paper focuses on improving learning solution operators for PDE on various geometries. First, the authors proposed a tokenization method  based on the eigenvalue of Beltrami-Laplacian operator $\Delta$ on boundary $\partial \Omega$, which characterizes the intrinsic geometry structure of $\partial \Omega$. By spectral clustering, the domain is decomposed into subregions. Second, the author proposed an architecture of neural operators that has two components. A coarse attention handles feature of the clustered points from the first step, and a fine attention handles features from original points.

Experiments include three datasets of challenging geometry shapes: Shape-Net Car, AirfRANS, Blood Flow. The proposed method outperforms all baselines.

STRENGTHS AND WEAKNESSES:
**Strength**

The idea of tokenizing points by spectral clustering is novel and reasonable. The paper is nicely presented and easy to follow.

**Weakness**

The proposed dual-attention structure causes overhead parameters and computation (see Table 9). One alternative would be simply using clustering as additional feature. Also, such clustering feature should naturally be utilized by the "slicing" structure of Transolver. Hence, I strongly recommend the authors to conduct an experiment of combining your tokenization technique with Transolver.

QUESTIONS:
N.A.

LIMITATIONS:
N.A.
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes SpiderSolver, a geometry-aware transformer that introduces spiderweb tokenization for handling complex domain geometry and irregularly discretized points. This method partitions the irregular spatial domain into spiderweb-like patches, guided by the domain boundary geometry. SpiderSolver leverages a coarse-grained attention mechanism to capture global interactions across spiderweb tokens and a fine-grained attention mechanism to refine feature interactions between the domain boundary and its neighboring interior points. Experimental results demonstrate that SpiderSolver consistently achieves state-of-the-art performance across different datasets and metrics.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. SpiderSolver is a geometry-aware Transformer-based PDE solver, which is designed to partition the domain into spiderweb-like patches utilizing physical and geometric knowledge to reduce the computation cost of attention.\n2. SpiderSolver adopts the multi-grained attention mechanism to capture the intricate physical correlation of complex boundaries and interior points. The fine-grained attention can extract the interact features between points of domain boundary and their near points in the interior of domain to model the intricate  influence of boundary shapes on solutions.\n3. Extensive experimental results are provided.\n\nWeaknesses:\n1. Please give a discussion on the robustness of spiderweb tokenization to the noise in boundary point coordinates.\n2. The time used for spectral clustering is not provided.\n3. The spiderweb tokenization generates patches that vary notably in size. Please give a discussion on the effect of variable patch size.\n4. In the experiment of generalization to shape variations of cars, if the shape of cars vary significantly, the patches obtained by spiderweb tokenization will vary in space across different car instances, how does the attention between patches learned by nearest shapes transfer to patches with varying position?  Please give a discussion on this.\n\nQUESTIONS:\nIn eq. 5, how is the initial point-wise feature $x^0$ computed? Since spiderweb token features $\\hat{f}^{l}$ is shared for each patch,  the initial point-wise feature must show significant difference among points.\n\nLIMITATIONS:\nThe proposed SpiderSolver is applied to PDE with boundary shapes of one type of objects. If the shape of objects vary dramatically, the tokenization transferred from the template may be unreliable. The spectral clustering for a new object with significantly new shape will be time-consuming. The attention transferred is unreliable too.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SpiderSolver, a geometry-aware Transformer designed to solve partial differential equations (PDEs) in irregular domains. SpiderSolver introduces spiderweb tokenization, which divides the domain into sections based on the shape's boundaries and distances, allowing for effective attention across different scales. When evaluated on five diverse PDE datasets with intricate geometries, SpiderSolver consistently achieves top performance and shows enhanced generalization compared to existing approaches.\n\nSTRENGTHS AND WEAKNESSES:\n## Strength\n1. Tokenizing irregularly structured domains, such as graph/mesh/point-cloud, is a challenging problem. The approach introduced here is novel to my knowledge and might be useful even outside of the problem of PDE solving.\n\n2. The proposed technique achieves better empirical results compared to the baselines.\n\n## Weaknesses\n\n1. Although the paper is mostly well-written and clear, the paragraph on \"Boundary Spectral Clustering\" is not explained clearly.\n\n2. It is crucial to follow up on all datasets of the major baselines (such as Transolver) for proper comparison. Datasets like Elasticity and Plasticity are not included.\n\n3. The proposed technique is not evaluated for zero-shot generalization on discretization (of the domain), meaning that the evaluation samples contain more or fewer points than the training samples.\n\n4. Table 9 indicates that the proposed model requires almost twice the time compared to Transolver.\n\n5. The results in Table 6 are not interpretable. It would be beneficial to add a single performance metric to clearly demonstrate its dependence on $m_I$ and $m_B$. Additionally, it is generally expected that as $m_I$ and $m_B$ increase, the model's performance will improve. However, in many cases, this is not true, and this crucial issue remains unaddressed.\n\nQUESTIONS:\n1. The results reported in Table 3 of the Transolver do not exactly match those in Table 5. Could you please clarify the differences in the evaluation process?\n\n2. Instead of aligning the boundary clustering for different instances of the object, what would happen if we performed boundary clustering independently for each instance? Would this approach improve performance? This could also serve as an interesting ablation study.\n\nLIMITATIONS:\nyes. The proposed model requires more time. It should be mentioned in the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SpiderSolver, a novel deep learning model designed to solve PDE on domains with complex geometries and irregular discretizations. The key innovation is a geometry-aware transformer architecture that addresses the challenges of applying standard transformers to such problems.\n\nThe technical contributions include:\n1. The proposed method first uses spectral clustering on the boundary points of the domain to create shape-aware partitions. Then, it divides the interior domain into \"spiderweb-like\" patches based on these boundary clusters and the distance of interior points from the boundary. This tokenization is designed to be more physically intuitive and computationally efficient than methods that treat all mesh points individually.\n\n2. To ensure consistent tokenization across different instances of a similar geometry (e.g., various car shapes), the authors propose an alignment strategy. They use optimal transport to match the boundary points of any given shape to a pre-defined template shape, transferring the clustering from the template to the new instance. \n\n3. SpiderSolver transformer employs a bi-level attention system to understand the coarse-grained and fine-grained features. \n\nThe authors evaluate SpiderSolver on five diverse datasets, including simulations based on Shape-Net Car/AirfRANS, blood flow in the human aorta, and canonical Navier-Stokes and Darcy flow problems. The results demonstrate that SpiderSolver consistently achieves SOTA performance, outperforming previous neural operators and transformer-based PDE solvers, particularly in its generalization to OOD cases.\n\nSTRENGTHS AND WEAKNESSES:\nS1. The core idea of \"spiderweb tokenization\" is a significant and intuitive contribution. \nS2. The dual-level attention mechanism is an intuitive yet effective way to mitigate the quadratic complexity of standard transformers. \nS3. The use of optimal transport for aligning boundary clustering is a sophisticated solution to handle variations in shape within a class of objects.\n\nW1. The authors acknowledge that the current formulation of SpiderSolver relies on a well-defined, closed boundary. This limits its applicability to problems with open-boundary domains or domains with multiple, disconnected internal boundaries that are not part of a single, continuous surface. While the Bounded Navier-Stokes dataset has multiple obstacles, the tokenization still relies on the minimum distance to any boundary component, which may not be optimal for capturing interactions between the obstacles themselves. \n\nW2. The aligned clustering method requires a \"template shape\" for each class of geometry. The paper does not fully detail how this template is chosen or how sensitive the model's performance is to this choice. An arbitrary or poorly chosen template could potentially lead to suboptimal tokenization for the entire dataset.\n\nW3. While the paper notes that spectral clustering is computationally manageable for the datasets used (e.g., around 3,600 surface points for the car), this treatment could become a bottleneck for problems with extremely high-resolution boundary meshes containing millions of points.\n\nQUESTIONS:\nPlease address the weakness points I mentioned.\n\nLIMITATIONS:\nPlease address the weakness points I mentioned.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}