SpiderSolver: A Geometry-Aware Transformer for Solving PDEs on Complex Geometries
Authors: Kai Qi
Fan Wang
Zhewen Dong
Jian Sun
Venue: NeurIPS 2025 poster ()
Keywords: Machine learning in PDE; Transformer architecture; PDE on complex geometry; Tokenization;
Primary area: machine_learning_for_sciences

Abstract: Transformers have demonstrated effectiveness in solving partial differential equations (PDEs). However, extending them to solve PDEs on complex geometries remains a challenge. In this work, we propose SpiderSolver, a geometry-aware transformer that introduces spiderweb tokenization for handling complex domain geometry and irregularly discretized points. Our method partitions the irregular spatial domain into spiderweb-like patches, guided by the domain boundary geometry. SpiderSolver leverages a coarse-grained attention mechanism to capture global interactions across spiderweb tokens and a fine-grained attention mechanism to refine feature interactions between the domain boundary and its neighboring interior points. We evaluate SpiderSolver on PDEs with diverse domain geometries across seven datasets, including cars, airfoils, blood flow in the human thoracic aorta, as well as canonical cases governed by the Navier-Stokes, Darcy flow, elasticity, and plasticity equations. Experimental results demonstrate that SpiderSolver consistently achieves state-of-the-art performance across different datasets and metrics, with better generalization ability in the OOD setting. The code is available at https://github.com/Kai-Qi/SpiderSolver.

SpiderSolver: A Geometry-Aware Transformer for Solving PDEs on Complex Geometries
Kai Qi",, Fan Wang" Zhewen Dong', Jian Sun
'School of Mathematics and Statistics, Xi an Jiaotong University, Xi' an, China
State Industry-Education Education Integration Center for Medical Innovations at Xi'an Jiaotong University {qikai1218, wangfan525, dongaheuen))attu. ( bataattu.jtttu xjtu, eet...du..ce. edu. en, jiansunoxjun edu.cn cn
Abstract
Transformers have demonstrated effectiveness solving partial differential equa- tions (PDEs). However, extending them to solve PDEs complex geometries remains a challenge. In this work, we propose SpiderSolver, a geometry-avee -ware transformer that introduces spiderweb tokenization for handling complex domain geometry and irregularly discretized points. Our method partitions the irregular spatial domain into spiderweb-like patches, guided by the domain boundary ge- ometry. SpiderSolver leverages a coarse-grainet attention mechanism to capture global interactions across spiderweb tokens and fine-grained attention mechanism to refine feature interactions between the domain boundary and its neighboring interior points. We evaluate SpiderSolver PDEs with diverse domain geometries across seven datasets, including cars, airfoils, blood flow in the human thoracic aorta, as well as canonical cases governed by the Navier-Stokes Darcy flow, elas- ticity, and plasticity equations. Experimental results demonstrate that SpiderSolver consistently achieves state-of-the-arr performance across different datasets and metrics, with better generalization ability in the OOD setting. The code is available at htttps:/gichub..com//// -asSs/iiiisiiiesSsrrrrrr
Introduction
Solving partial differential equations (PDEs) is fundamental many computational problems in science and engineering. Classical numerical methods involve discretizing computational domains and solving the resulting algebraic systems. However, for domains with complex boundary geometries and irregular discretization, these methods require complex mesh generation and incur high computational costs. For example, computing the drag force on a car (Figure requires solving the Navier-Stokes equations with a car-shaped boundary. Classical methods discretize the computational domain into irregular meshes or points (e.g., over 30,000 points for the Shape-Net Car dataset), leading to high computational complexity and cost.
In recent years, there has been growing interest in applying deep learning methods solve PDEs, such as PINNs [[], neural operators 2 3 4. etc. However, most of the current methods deal with regular computational domains. For example, the well-established FNO [3] solves PDE in rectangular domains with uniform grids due to its implementation via the Fast Fourier Transform. To efficiently solve PDEs with irregularly shaped domains, Enometrr-IIformes Neural Operator (GINO) (5) and Geo-FNO [6] are proposed based on deforming the irregularly shaped domain into uniform latent mesh. Such approaches encounter difficulties with mesh transformations for complex geometries of the computational domains. The Graph Neural Operator (GNO) [2] adopts the Graph Neural Network 7 as a backbone by formulating the kernel integral operator as message passing on graphs,
Equal contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Dey
Wist
(n) Shape-Net Car
(b) AirIRANS
(c) Blood Flow
Figure 1: Visualization of datasets. For Shape-Net Car and AirRANN,, we estimate the surface pressure on the car/airfoil and the surrounding air velocity based on their shapes, aiming to predict the drag and lift forces for a driving car or flying airplane. The object surfaces of cars, airfoils, and blood vessels serve as PDE domain boundaries, with the interior regions surrounding the car/airfoil surfaces or lying inside the blood vessel.
which can be directly applied to irregular meshes. 3D-GenCA (8] introduces a pre-trained 3D vision model [9] for GNO, to encode the complex boundary of the domain. However, it does not fully consider the complex physical relationships between boundary and interior mesh points.
Recently, Transformers [10), a widely used backbone in deep learning, has been applied to solve PDEs, which can also be directly applied on irregular meshes. FactFormer [T, MINO [2], and Galerkin Transformer [13] are based on the idea that the dot- dot-product attention can be considered as an approximation of an integral transform with a non-symmetric learnable kernel function, which relates Transformers to the FNO. These transformers have demonstrated promising performance for solving PDEs with complex geometry. However, massive mesh points lead to the huge computational overhead of Transformers because the canonical attention in Transformers has quadratic complexity, and brings challenges for Transformers to capture the complex physical relations between irregular mesh points. Transolver [14] proposes Physies-Attention mechanism that decomposes the dis- cretized domain into a series of learnable slices, with mesh points exhibiting similar physical states assigned to the same slice. Notably, the slices are learned without explicitly utilizing the geometric information of the domain.
In this paper, we focus on designing transformer-basee PDE solver for domains with complex boundary geometries and irregularly discretized points. The main challenges include efficient tokenization over irregular computational domains and the integration of domain and boundary geometry into the network design. Following this motivation, we propose SpiderSolver, a geometry- aware Transformer. SpiderSolver uses spiderweb tokenization to partition the domain into spiderweb- like patches, guided by spectral clustering of the boundary and the distance of interior points from the boundary. Spiderweb tokenization partitions physical space in a more fundamental and physically intuitive way, achieving a trade-off between computational efficiency and capturing the physical interactions of spatial points. Based on spiderweb tokenization, shown in Figure 2 SpiderSolver integrates coarse-grained and fine-grained attention to capture the physical relationship between points of interior domain and boundary surface. We evaluate our SpiderSolver design tasks and one blood simulation task, as well as canonical cases governed by the Ntvker-Stokes and Darcy flow equations. These tasks are challenging since they require the model to handle various complex boundary geometries. The experiments show that our SpiderSolver achieved state-of-the-an results compared other neural operators and Transformer-baue based PDE solvers for these tasks.
two industrial-level
Overall, our contributions are as follows. First, we introduce a novel complex computational domain partitioning method, Spiderweb Tokenization, which divides the domain into spiderweb-like patches. Second, we propose SpiderSolver, a geometry-aware transformer for solving PDEs with complex boundary geometries and discretization. SpiderSolver integrates boarse-grained and fine-grained attention to capture the physical relationship between inner space and boundary surfaces. Third, SpiderSolver surpasses the state-of-the-ar methods in the PDE solving tasks for car and airfoil design, the blood flow dynamics in the human thoracic aorta, as well as two fundamental PDE tasks.
2 Related Work
The deep learning methods have been widely applied to solve PDEs, which can be roughly categorized into two paradigms. The first paradigm, such as PINNs M. Deep Ritz 15), etc, is to approximate the solutions of PDEs by neural networks and formalize the physical constraints (including equations,

---

initial and boundary conditions) as objective functions to optimize network parameters. This kind of method requires the exact formalization of PDEs and needs to retrain the network for new PDEs. Another paradigm is operator learning, e.g., DeepONet [4] and FNO 3 It is to learn the nonlinear mapping (i.e., operator) between the function space of parameters and the function space of the PDEs' solutions. FNO [3] approximates the solution operator by the kernel integral operator and calculates it in the frequency domain by the fast Fourier transform (FFT). Afterward, a series of variants of FNO are proposed. F-FNO [16] enhances model efficiency by employing factorization in the Fourier domain. U-NO 17 and U-FNO [8] combine FNO and U-Net for multiscale problems. However, most of these methods solve PDEs with regular computational domains.
For solving PDEs with irregularly shaped domains, Geo-FNO 66] and GINO [5] are based on the idea of deforming the irregular input domain into a uniform latent mesh on which the FFT used in FNO can be applied. However, such mesh transformations are difficult for complex geometries of the computational domains, such as car-shaped domain, resulting in performance degradation. Graph Neural Operator (GNO) 2 formulates the FNO's kernel integral operator as message passing on graphs leveraging Graph Neural Networks 7 to construct neural operators. However, it should be noted that graph kernels are insufficient in their capacity to capture global information.
Transformers [0] have also been introduced as a backbone to solve PDEs with irregularly shaped domains. MINO [12, FactFormer [T]] and Galerkin Transformer [S] are based on the idea that the dot-product attention can be considered as an approximation of an integral transform with a non-symmetric learnable kernel function, which relates Transformer to the FNO. Vito [99] combines the vision transformer (20) and the U-net [2 to construct the neural operator. To overcome the
quadratic complexity of attention, GNOT 22 and ONO 23 utilize the well-established linear Transformers, such as Reformer [24]. Performer [25] These methods directly apply attention to
mesh points, which is computationally prohibited when the number of points is large. Transolver [[4] decomposes the discretized domain into series of learnable slices, in which mesh points under similar physical states are assigned to the same slice. Then Transolver applies attention to these
learnable slices to learning intrinsic physical relations.
In contrast, SpiderSolver
geometry-aware Transformer- based PDE solver, which is designed to partition the domain into spiderweb-like patches utilizing physical and geometric knowledge to reduce the computation cost of attention. Besides, SpiderSolver adopts the multi-grained attention mechanism to capture the intricate physical correlation of complex boundaries and interior points.
3 Our Proposed SpiderSolver
Problem setup. We consider the partial differential equations (PDEs) defined over a domain Q E R4. where d represents the dimensionality of the space. In this work, the domain Ω has general irregular geometry and boundary, typically discretized into N mesh points, represented as G E R xd Specifically, G N { where Ia = {9 N2 represents the N, interior points of $ (off-boundary points), and B& { Ne denotes the Ng boundary points of Ω2 and NB + N, N. Our goal is to learn a non-linear operator that outputs the physical field u to approximate physical quantities over the geometry G.
For PDEs with domain boundaries in general irregular geometries (e.g.. the boundaries shown in Figure []. the physical quantities at different locations within the domain are influenced by their proximity to the boundary and the geometry of the boundary itself. In automotive aerodynamics, the flow field exhibits stratification around the vehicle. Near the car surface, airflow adheres to the surface geometry, while at greater distances, the velocity gradually aligns with the freestream. The acrodynamic effect is influenced by the car's surface curvature and angle of attack. Horizontal surfaces, such as the roof, have less impact on the flow, whereas regions with larger angles, such as the front windshield, significantly alter both the flow direction and speed.
Based on the above observations, we propose SpiderSolver, a geometry- aware transformer, specifically defined as fast PDE solver for solving PDEs over the domains with irregular geometries. As shown in Figure? 2 SpiderSolver integrates spiderweb lokenization (Figure (a)) into transformer architecture. As will be presented in Section 3.1 spiderweb tokenization partitions the domain of PDE into spiderweb-like patches considering the geometry of the domain boundary, and each patch is used to define a token. Built upon the spiderweb tokenization, we design transformer (detailed in Section
3

---

SpiderAieen Bleck
SpiderAnen
SpiderAmes
Einbekking
Spulerweh
Toketuzation
LayeNomm, FesiForwand
Tokes
Weighted Token
Point
Allantion
Apuregation
Aseation
Cost genneee
Fine gruined- Allention
Car surface and
Ateetion
Layerhom
Spideweb Tokenization
Atsentio
grained Atlention
SpiderAttin Block
Figure 2: Overall architecture SpiderSolver. Spiderweb Tokenization partitions the domain into Spiderweb-like like patches. Coarse-graine grre-erriddd Attention interacts the features over the spiderweb tokens. Weighted Token Aggregation updates the point-wise features by the weighted combination of token features. Fine-grained Attention enables interactions between points of boundary and near boundary.
3.2 consisting of cascaded attention blocks, and each block is composed a paarse-grained attention (Figure (b)) over the spiderweb tokens and a fine-grained attention (Figure (c) to interact features between points of domain boundary and their near points in the interior of domain, This transformer is the basis of our SpiderSolver, which is learned to output the PDE solutions.
3.1 Spiderweb Tokenization over PDE Domain
Overview. To solve PDEs with complex boundary geometries, introduce geometry-awant tokenization method as a foundational step in constructing our transformer. The aim is to quantize the computational domain Ω with interior point set Ig and boundary point set BG into non-overlapping sub-regions adaptive to the geometry of domain boundary (e.g., the object surfaces in Figure [T. This process begins with spectral clustering for the domain boundary OL quantize the domain boundary to sub-regions. Then the inner space of the domain Ω2 is partitioned based on the boundary clustering and the distance of interior points the boundary. We term this approach "spiderweb tokenization" due to the spiderweb-like structure of the resulting divided sub-regions, illustrated in Figure 3
Template Shope
Templac Spectal
Optraal tatspoet-baacl Point
Special
Other Skeps
Aligned Clusteing
Figure 3: Spiderweb-like sub-region structure from spiderweb tokenization. Colored curves the surface illus- trate spectral clustering-based partition.
Figure 4: Optimal transport-based alignment for spectral clustering. (a) Aligning surface points of a shape to the template shape for clustering. (b) Comparison of individual and aligned clustering results.
Boundary spectral clustering. Spectral clustering over the boundary leverages the eigenvalue problem of the Laplace operator A, which can defined on non-Euclidean geometry for capturing its geometric and topological propertics [26). For the continuous domain of boundary surface OΩ C R², the eigenvalue problem is:
-Au = Xu, in 02,
(1)
subject to appropriate boundary conditions. The eigenvalues and eigenfunctions u characterize the intrinsic geometric structure of the boundary aΩ, enabling its decomposition into geometric decomposed sub-regions. These eigenvectors u are used to embed the data into a low-dimensional space. Specifically, the Laplacian operator be approximated by constructing the affinity matrix

---

over nearest neighbors (10 neighbors) graph using points or mesh vertices on the domain boundary. Then we use k-means clustering on the eigenvectors of the normalized Laplacian matrix associated with the my smallest eigenvalues, to group an into m clusters. Spectral clustering leverages the graph structure of surface points to capture global, shape-aware connectivity beyond local geometry. See Appendix A for implementation details of spectral clustering.
Aligned boundary clustering for different instances of object surface. As shown in Figure 4 (b) and Figure To (Appendix [B). individual clustering of different instances of an object results in unaligned clustering. We devise an aligned clustering approach by transferring the clustering of template shape to the remaining instances of the boundary shape. As shown in Figure 4 (a), we first align all other shapes to a randomly selected reference shape using the entropic regularization optimal transport problem solved by the Sinkhorn algorithm [27]. We then average these aligned shapes to obtain a template shape, on which spectral clustering is performed. Finally, the clustering labels from the template shape are transferred to the other instances of the object. Here, the reference shape only used for alignment, while the template shape serves as the basis for spectral clustering. Therefore, given a new instance of object, we only need to align its points to template shape points to derive its point clustering. This above strategy to align PDE boundaries to the boundary template shape can accomplish the varying shapes of boundary geometry for a given type of geometry, e.g.. car, airfoil. Please refer to Figure 9 in Appendix
B for the example of template shape.
Spiderweb tokenization. Based on aligned boundary clustering, we further divide the inner space of Ω into sub-regions Tpg with p = 1, ...., m and q = ......., my, as shown in Figures 3 and
2 (a). Spectral clustering applied to the boundary BG yields mB clusters, therefore Bg
Cp
with Cp denoting the p-th cluster of the boundary. As shown in Figure 3 interior PDE domain is divided into sub-regions, and each sub-region Tyg includes the interior point with its closest point on
domain boundary belonging to p-th boundary cluster, and its signed distance function (SDF) values to domain boundary within a range (dg-1,dg). dg). with dg = O. This region is expressed as:
Tpg {g Ic SDF(g) (dg-1, dgl. 83 E Cp. 83
arg min llg syll] [1lls eG
(2)
We determine the range of (dg- dy) by ensuring an equal number of points falling within each interval (dg-1,dg) dg]. In this way. we partition the whole PDE domain into M = mmm, + mB sub-regions with mms sub-regions in the interior domain and my sub-regions the boundary. Each sub-region is taken as the domain of a token.
Notes: The spiderweb- spiderweb-tokenization method is inherently geometry- aware, explained below. Firstly, it utilizes the geometric information of boundary to partition the domain effectively. Secondly, by optimal transport-based point alignment, it ensures that the clustering of object surfaces, i.c., domain boundaries, remains consistent across different instances. This alignment enables a well-structured tokenization of PDE domain, even when boundary shapes vary. As result, the geometrically aligned tokens allow transformers to function as PDE solvers that inherently account for boundary geometry.
3.2 Transformer Design for SpiderSolver
Based on spiderweb tokenization (Figure 2 (a)). we define SpiderSolver (Figure 2 as a transformer defined over spiderweb tokenization, with SpiderAtten layers integrating coarse coarse-grained ( and fine-grained attentions (Figure The coarse-grained attention facilitates interactions among spiderweb-like tokens. The fine-grained attention serves as a finer-level complement, capturing interactions between boundary points and their near interior points adjacent to the boundary.
As shown in Figure 2 we summarize the overall process of SpiderSolver. First, a linear embedding maps the input G E R- or with the observed physical quantity, to initial feature x" e R NxC. Next, x" passes sequentially through the L SpiderAtten blocks (Figure (d)). The [-th (I [0, L 1)) SpiderAtten block is defined as:
Spiderweb-Tokkeizztton (x'),
x +1 = x' + T2P (Coarse-AT AT ["))x]] + Fine-AT (LN (x')),
(3)
++ FeedForward (LN 1x) x
where the "Coarse-AT refers to coarse-grained attention, "Fine-AT" refers to fine-grained attention, "LN" denotes layerNorm, and x x NxC . MxC, The T2P operator transforms the
5

---

token features back to update the point-wise features. The "FeedForward" operator consists of two linear layers with non-linear activation in between. The operation Spiderweb- Spiderweb-Tokenizztiin Tokenization (xx)) defined as the concatenation of average pooled features of x' over spiderweb-toker domains Tpq: for E [I, m. E [1, my], together with m additional feature by average feature pool- ing over mg boundary clusters. This coarse-level tokenization facilitates the fast self-attention computation from the interactions of N points to M tokens.
After L SpiderAtten blocks, linear embedding layer over x" is applied to obtain the output physical quantities, i.e, the PDE solution. We next introduce the operators of Coarse-AT(.), T2P(:)) Fine- AT(-) of Equation
3.2.1 Coarse-grained Attention (Coarse-AT)
The coarse- suase--rained attention (Figure (b)), denoted as Coarse-AT in Equation (3). is defined over the M tokens of the spiderweb tokenization of the PDE domain. For the I-th SpiderAtten block, with the token-level feature E RMCC we apply the self attention, denoted as A(-):
Fd = A(()') [Ai (f'); ( (F))
where A((f') Softmax (Q:K7 / /d Vi,
to conduct multi-head attention, where dy denotes the dimension of key vectors, [-] denotes concate- nation, T; = Linear(f', [f', W4.) for T; E {Qi, Vi 1,.., and h denotes the number of heads. "Linear" refers to a linear layer and generates Q, K, V, E IR MxC in attention. Note that, the self-attention is conducted over the spiderweh tokens instead of all the interior points of PDE domain. Thus, the output the operator Coarse-AT is f', i.e., the coarse-grained attention feature.
3.2.2 From Token to Point-Wiss Features (T2P)
After coarse-grained attention, the attended token features f' Coarse-AT(f)) will be transformed into point-wise features and added to the corresponding point-wise features x' Let x/ E R-VxC. x RNNxx be the point-wing wise features x" in the interior and the domain boundary respectively. Let f/ R.mamyx xC and F/1 E m be the spiderweh token features f of the interior and boundary tokens respectively. The updated point-wise features are computed based on the weighted token aggregation, which globally aggregates all the token features to compute the point-wise update, formulated as:
Linear(x')), st Liiner(x()), (5) x = -/"" 2/1 - S/2T% + E. where of = LLiner(x))),
-
where the three "Linear" layers are respectively with learnable parameters W W/ and W and generates 7/ E R NNNixmmss L E N x . Na xmpmm weighting matrix to combine spiderweb token features. Therefore, the operator T2P outputs updated features xg) It facilitates the point-wise feature updating by the coarse coarse-grained attention features over spiderweb tokens.
3.2.3 Fine-grained attention (Fine-AT)
The fine- mne-grained attention (Figure 2 (c)) is designed to enhance the interaction between features of points located on boundary and points near boundary. This design is motivated by the observation that points get closer boundary, their physical characteristics are increasingly influenced by the boundary geometry. In practical applications, crucial physical quantities, such as the drag coefficient of car and the lift coefficient of an airfoil are derived from the features
near the boundary.
To effectively capture these counday--relatee interactions, fine-grained attention is introduced to facilitate feature interactions between the domain boundary points B& and their nearest interior domain neighboring points in the set TB = {g E La (SDF(g) E di]. The corresponding features of these boundary and near-boundary interior points in TB are concatenated to form combined feature representation XF. The attention then applied to this representation to capture the underlying physical dependencies by A(xF), which is the output of the operator Fine-AT.
3.2.4 Network Training and Testing
The proposed transformer contains the learnable parameters in the two embedding layers and SpiderAtten blocks in Equation 3 The network is trained based on the relative L L2-norm loss

---

between the network output and the ground- truth PDE solutions over the training set 14. In the training phase, spiderweb tokenization is performed over the training dataset as a data pre-processing procedure. In the testing phase, given a new instance of the PDE geometric domain and/or boundary condition, the PDE domain is divided into token sub-regions by aligned spectral clustering, and then the SpiderSolver outputs the PDE solution based on the spiderweb tokenization.
4 Experiments
We evaluate SpiderSolver on five datasets spanning industrial, biomedical, and fundamental PDE tasks. The template shapes of Shape-Net Car and AirfRANS are visualized in Figure Appendix As the Blood Flow, Bounded Navier-Stokes and Darey Flow datasets have fixed geometries, optimal transport-based alignment is not required. See Appendix more details on five datasets.
Shape-Net Car 333 consists of 889 simulated samples based on Reynolds-Averag Navier-Stokes equations, with car shapes from the ShapeNet "car" category (34] Each sample includes velocity and pressure fields solved via a finite element method over 32,186 186 mesh points, Using irregularly discretized car and surrounding space as input, the model is trained to predict these fields, from which the drag coefficient is subsequently derived. The shapes of cars ran- domly selected from Shape-Net Car are visualized in Figure 7 in Appendix B
Table 1: Results on Shape-Net Car and AirfRANS datasets. Vol: error of surrounding physics field; Surf: error of surface physics field. CD. C₁.: error of drag and lift coefficients; PD, PI. Spear- man's 's rank correlation of drag and lift coefficients.
Shape-Net Car
AirRANS
Methods
Vol 4 Surf 4 Cp 4 Pn
Vol
Surf 4
C2 4 Pr.
Simple MI.P
0.0512 0.1304 0.0307 0.9496
0.0081 0.0200 0.2108 0,9932
G-SAGE 28
0.0461 0.1050 0.0270 0.9695
0.0087 0.0184 0.1476 0,9964
PointNet
0.0494 0.1104 0.0298 0.9583
0.0253
0.0996 0.1973 0,9949
G-U-Net
0.0471 0.1102 0.0226 0.9725
0.0076
0.0146 0.1677 0,9944
MG-Net
0.0354 0.0781 0.0168 0.9840
0.0214
0.0387 0.2252 0.9945
GNO 2
0.0383 0.0815 0.0172 0.9834
0.0269
0.0405 0.2016 0.9934
Galerkin 13
0.0339 0.0878 0.0179 0.9764
0.0074
0.0159 0.2336 0.9957
Geo-FNO 6
0.1670 0,2378 0,0664 0.8280
0.0361
0.0820 0,6614 0.9257
GNOT 22
0.0329 0.0798 0.0178 0.9833
0.0049
0,0152 0,1992 0,9942
GINO S
0.0386 0.0810 0.0184 0.9826
0.0297
0.0482 0,1821 0.9958
3D-GeoCA
0.0319 0.0779 0.0159 0.9842
Transolver 4
0.0228
0.0793 0.0129 0.9916
0.0025
0.0080 0.1026 0,9977
SpiderSolver
0.0210 0.0738 0.0100 0.9928
0.0017 0.0043 0.0741 0.9988
AirfRANS (35] comprises 1,000
simulations over 32,000-point meshes of 4- and S-digit NACA airfoils, with variations in shape, Reynolds number, and angle of attack. Using irregularly discretized airfoil and surrounding space input, the model is trained to predict velocity, pressure, and viscosity fields, from which lift
coefficient is computed. The shapes of airfoils are visualized in Figure & in Appendix B
Blood Flow dataset [36] consists of 500 simulations of blood flow in a fixed human thoracic aorta geometry, by the Navier-Stokes equations. Each sample varies in inlet and outlet conditions, and is discretized over a tetrahedral mesh with 1,656 spatial nodes and 121 time steps. The model is trained to predict the velocity field of the flow, using pressure the inlets and outlets.
cwdfiicntt 2.2933)
(Diza coeffiicet 9,2544)
Trmooler
(Dvig wodfiiiian 0.2000
Dum
Thmudorr (Lil codflciktt 0.063)
Dare collooolii 09168) (Rdaaiee dy 60210) (Relatvee &g 0.0100)
Darc
Trush (Lit oxcliinn BUsDAR
Giann Tr 0.46)
Figure 5: Examples from three datasets. The first column shows the ground truth. In (a), (b), and (c). the first row presents the predictions by Transolver and our model, while the second row displays the point-wise La norm of the difference between ground truth and predictions.
Bounded Navier-Stokes dataset with multiple separate boundaries (37 simulates 2D fluid flow through pipe containing several fixed pillar-like obstacles, resulting in multiple disconnected boundaries. Despite the presence of separate boundaries, the SDF at each point is uniquely defined as the minimum distance to all boundary components. See Appendix C for more details on datasets.
7

---

Darey Flow dataset [3] considers the steady-state of the 2-d Darey Flow equation on the unit box. In the Darey Flow dataset, we take the spatial outer rectangle boundary as the boundary to compute clustering and SDF for tokenization. See Appendix C for more details on datasets.
Elasticity and plasticity datasets. Elasticity 66 models unit cell with arbitrary void, governed by the incompressible Rivlin-Saunders constitutive law. Plasticity [6 considers block impacted by a rigid die, governed by an clasto-plastic constitutive model with time-dependent deformation.
Implementation details. To ensure that our model parameters are comparable to other
Transformer-baseer models, such as Transolver, we set the number of layers as 8 and the channel of hidden features as 256 or 512, depending the number of observed quantities of input data, All
Table 2: Results on Blood
Flow dataset. Methods
Velocity +
experiments are performed on a GeForce RTX 4090 GPU. See Appendix [for more details.
Simple MLP
0.3080
DeepONet 4
0.8926
Compared methods and metrics. We compare SpiderSolver with more than 18 baselines. For the Shape-Net Car and AirfRANS, we evaluate the estimation error of physical fields in the PDE domain (Vol), domain boundary surface (Surf), drag coefficient (Cp) and lift coefficient (CL) using the relative L2 except that we follow [35 [4] to MSE for the Vol and Surf on the AirfRANS dataset. We additionally use Spearman's 's rank correlations of drag and lift coef- ficients, respectively in the Shape-Net Car and AirfANN datasets. For the other datasets, we use the relative 122 error as the metric. See Appendix E for details.
POD-D 33
0.3742
Gco-FNO 6
0.1209
GNOT 32
0.0411
NORM 36
0.0453
Geo-FNNO 6
0.1209
3D-GcoCA 8
0.2863
GINO S
0,1864
Transolver [4]
0.0438
SpiderSolver
0.0322
4.1 Main Results
Results on Shape-Net Car. As shown in Table SpiderSolver outperforms various methods. The Spearman's 's rank correlation indicates that our predicted drag coefficients better match the true ranking. Figure MT in Appendix B shows an example of spiderweb tokenization by SpiderSolver.
Results AirIRANS. As shown in Table 1 SpiderSolver reduces the MSE of the volume and surface physics fields by 32.0% and 46.3%, compared to Transolver. Figures 12 and i3 in Appendix B show an example of spiderweh tokenization by SpiderSolver and the patches learned by Transolver.
Results on Blood Flow. As shown in Table 2 SpiderSolver achieves state-of-the ahe performance. Figure visualizes the predicted velocity fields.
Results on Bounded Navier-Stokes and Darey Flow, SpiderSolver outperforms various methods on both Bounded Navier-Stokes and Darey flow datasets. See Tables [3 and [4] in Appendix [ for results. Darcy Flow dataset is defined on reg- ular quadrilateral domain, and both the Bounded Navier-Stokes and Darcy Flow datasets use uni- form grids. Although SpiderSolver is designed for complex boundaries and irregular grids, it re- mains applicable to such settings. Figure 14 in Ap- pendix B shows two examples of spiderweb tok- enization on Bounded Navier-Stokes dataset. For
Table 3: Generalization in OOD AirRRANS.
Reynolds OOD Angles OOD
Methods
Cr. 4 pr. t Cr. 4 PI. 1
Simple MLP
0.6205 0,9578 0.4128 0.9572
G-SAGE 28
0.4333 0.9707 0.2538 0.9894
PointNet 29
0.3836
0.9806 0.4425 0.9784
G-U-Net 33
0.4664
0.9645 0.3756 0.9186
GNO 2
0.4408 0.9878 0,3038 0.9836
Galerkin 13
0,4615 0.9862 0.3814 0.9821
GNOT 32
0.3268
0.9865 0.3497 0.9863
GINO 5
0.4180
0.9645 0,2583 0.9923
Transolver 14
0.3889
0.9911 0,2490 0,9940
the Bounded Navier-Stokes dataset, SpiderSolver applies spectral clustering to all obstacle surfaces, where coarse- gaarse- attention directly captures in- teractions among tokens. As shown in Figure 14 (Appendix B the resulting interior partitions align well with obstacle geometry, enabling attention maps to encode mutual influences between obstacles. Ablation results further show that merging obstacle surface tokens reduces accuracy (relative L2 error 0.0432), while explicit obstacle token modeling in SpiderSolver achieves superior performance (relative La error 0.0376).
SpiderSolver 0.2291 0.9922 0.1062 0.9941
Results on elasticity and plasticity datasets As shown in Table 15 in Appendix F SpiderSolver consistently outperforms all basclines, achieving the lowest relative L2 error on both datasets.
8

---

Generalization to Reynolds number and airfoil angle variations on AirfRANS. We assess the generalizability of SpiderSolver in two scenarios using AirRANN [55. (1) Reynolds extrapolation (Reynolds OOD): the training set includes samples with Reynolds between 3 and 5 million, while the Reynolds of the test set spans 2 to and 5 6 million. (2) Angle of attack extrapolation (Angles OOD): the training set covers angles from - 2.5 to 12.5", and the test set includes angles from - to 225 and 12.5 to 15". Table 3 shows the metrics for the OOD experiments on
AirIRANS dataset. The results demonstrate that our proposed SpiderSolver achieves consistently better outtrf-distribution generalization on the AirfRAN dataset.
Generalization to shape variations of cars on ShapeNet-Car. We compute the average point-wise Euclidean distance from each car shape to the tem- plate shape, and then select the 200 nearest shapes for training and the 100 farthest shapes for testing.
Table 4: Generalization to shape variations of cars on Shape-Net Car dataset.
Shape-Net Car
Methods
Vol +
Surf L
Cp +
PD t
The results are reported Table 4 SpiderSolver achieves good distrbbbbtiiiiiinnnnnno generalization to shape variations of cars on ShapeNet-Car, In Ap- pendix D we analyze the diversity of car geometries, and we further visualize how prediction error varies with different distances to the template shape, but observe clear correlation between model accuracy and shape distance to template shape.
Transolver [4] 0.0660 0.191 0.0735 0.9142
SpiderSolver 0.0510 0.161 0.0550 0.9222
4.2 Ablation Study and Model Analysis
Effects of key components of the network. We conducted the ablation study on the key components in Table Referring to Equation (3) "w/o Coarse-A" denotes our SpiderSolver without the Coarse- AT and the T2P operator, and "w/o Fine-AT" denotes SpiderSolver without Fine-AT operator. "T2P-local" refers to T2P operator in Equation (5) that is computed locally in each spiderweb token. Specifically, in "T2P-local", Equation (5) is computed locally for each spiderweb token to generate the updated feature for the points belonging to the token. As shown in Table []] removing Coarse-AT and Fine-AT considerably degrades the model's performance. Compared with removing Fine-AT, the performance drop when Coarse-AT is removed is more remarkable, highlighting its crucial role. Appendix Fl compares the runtime of SpiderSolver with and without Fine-AT.
Table 12
Table 5: Ablation study of key components of SpiderSolver.
Shape-Net Car
AirRANS
Blood Flow
Ablation methods
Vol 4
Surf 4
Cn 4
pD f
Vol 4
Surf +
Cr. 4 pr. 1
Velo +
T2P-local
0.0218 0.0741 0.0114 0,9881 0.0032 0.0055 0.0717 0.9989
0.0345
w/o Fine-AT
0.0236 0.0896 0.0136 0,9891 0,0022 0.0099 0.1013 0.9986
0.0347
w/o Coarsc-AT
0.0324 0.0962 0.0204 0.9765 0.1258 0.0101 0.1576 0,9962
0.0634
w/o Coarse-AT and Fine-AT
0.0645 0.2390 0.0629 0.8585 0.1917 0.5314 0.1793 0.9915
0.0874
SpiderSolver
0.0214 0.0725 0.0096 0.9957 0.0016 0.0039 0.0665 0.9984
0.0322
Ablation study reference shape selection. The reference shape serves only provide consistent indexing, and it does not affect the geometry of the resulting template shape. As shown in T model exhibits strong robustness to the choice of reference shape Shape-Net Car, consistently outperforming Transolver across different reference settings. Furthermore, spectral clustering results across five reference shapes demonstrate high consistency, with Dice of 0.9975 and IoU of 0.9951. Dice and loU are computed based on the intersection and union of the five clusterings.
Analysis of seper-parameters on validation set. We compare the performance and number of parameters of models with varying feature channels C and network layers in Figure 6 and the varying my and my in Table 8 Ours consistently outperforms Transolver across different C' and L.
Robustness to noisy boundary points. To evaluate the robustness of spiderweb tokenization noisy boundaries, we introduced Gaussian perturbations with varying noise levels to the car surface point cloud. As shown in Table 7 the spectral clustering and SDF-based partitioning preserve domain structure under perturbations. Furthermore, model evaluation on Sha Car confirms that the robustness of patch partitioning directly contributes to stable performance against noisy boundary conditions.

---

Table 6: Ablation study on selection of the reference shape for Shape-Net Car.
Shapes
Vol 4
Surf L
CD +
PD t
Reference shape I
0.0215
0.0730
0.0095
0,9934
Reference shape
0.0212
0.0721
0.0103
0,9935
Reference shape 3
0.0215
0.0725
0.0105
0,9921
Reference shape
0.0210
0.0727
0.0103
0.9934
Shape used in Table
0.0210
0.0738
0.0100
0,9928
Mean
0.0212
0.0728
0.0101
0,9930
Standard Deviation
0.0002
0.0006
0.0003
0.0005
Table 7: Robustness of spiderweb tokenization and model performance under noisy boundary perturbations on Shape- Net Car (ar is the noise standard deviation).
σ
Dice
IoU 1
Vol |
Surf
Cp L
PD 1
0.0001
1,0000
1.000
0.0210
0.0738
0.0100
0.9928
0.001
0.9994
0.9987
0,0217
0.0739
0.0103
0.9924
0.01
0.9868
0.9743
0.0230
0.0790
0,0119
0.9919
0.03
0.9534
0.9135
0.0310
0.0829
0.0135
0.9901
Table 8: Ablation study of SpiderSolver with varying my and my on the validation set.
budeher Speletobea (Dur))
hendaver So venda 1Dusl
AidRANS
Blood Flow
(m,, mm)
Vol
Surf 1 C2 4 PL
Velo +
2)
0.0026 0.0130 0.0885 0.9987
0.0372
2)
0.0021 0.0101 0.0818 0.9988
0.0366
(2, 4)
0.0019 0.0064 0.0751 0.9989
0.0359
Figure 6: Performance of our model and Transolver
(6, 2)
0.0020 0.0079 0.0624 0.9990
0.0352
on the Blood Flow dataset with varying channels C
(2, 6)
0.0019 0.0075 0.0583 0.9988
0.0350
and layers L (point size о model size).
Computational cast. On the Shape-Net Car dataset, SpiderSolver has 4.05M parameters and the
network inference time for one PDE is 0.058s, compared with the 3.86M and 0.022s respectively by previous state-of-the of-te--f--h-arr art method Transolver [14]. Note that it takes around 50 minutes [33] required by the traditional k-epsilon turbulence simulations. See Appendix E for details on inference time.
Conclusion, Limitations and Future Work
Conclusion and impact statement. This paper proposed novel transformer SpiderSolver, to solve PDEs with complex domain geometry. SpiderSolver based on the proposed spiderweb tokenization and integrates coarse-grained and fine-grained attention. Experiments on five datasets demonstrate its superiority. This work has made a fundamental contribution to the transformer architecture design in scientific computation, and may have impact the fast computation of numeric solutions to PDEs with complex boundaries, in the applications science and engincering problems.
Limitations and future work. SpiderSolver depends on the boundary and is currently not applicable PDEs with boundayy domains. Moreover, currently applied PDE with boundary shapes of one type of objects. In future work, we plan to extend the framework to multiple classes of shape boundaries, possibly by making the network parameters aware to different geometric classes. We also plan to extend it to be equivariant to the geometry transform of PDE boundary and domain.
Acknowledgement
This work was supported by NSFC 12125104, 12426313, 12326615, Key-Area Research and Development Program of Guangdong Province (2022B0303020003)
10

---

References
[1] Maziar Raissi, Paris Perdikaris, and George E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
[2] Zongyi Li, Nikola Kovachki, Kamyar Arizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Graph kernel network for partial differential equations. In ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations, 2020.
[3] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differen- tial equations. In International Conference on Learning Representations, 2021.
[4] Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universial approximation theorem of operators. Nature Machine Intelligence, 2-299, 2021.
[5] Zongyi Li, Nikola Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Otta, Moham- mad Amin Nabian, Maximilian Stadler, Christian Hundt, Kamyar Azizzadenesheli, et al. Geometry-informed neural operator for large-scale 3d pdes. In Advances in Neural Information Processing Systems, 2023.
[6] Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries. Journal of Machine Learning Research, 24(388):1 2023.
[7] Justin Gilmer, Samuel S Schoenholz, Patrick Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International Conference on Machine Learning, 2017.
[8] Jingyang Deng, Xingjian Li, Haoyi Xiong, Xiaoguang Hu, and Jinwen Ma. Geometry- conditional adaption for surrogate models of large-scale 3d pdes on arbitrary geometries. International Joint Conference on Artificial Intelligence, 2024.
[9] Le Xue, Mingfei Gao, Chen Xing, Roberto Martin-Martin, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Leaming unified representation of language, images, and point clouds for 3d understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1179 179--189, 1189, 2023.
[10] A Vaswani. Attention is all you need. In Advances in Neural Information Processing Systems, 2017.
[11] Zijie Li, Dule Shu, and Amir Barati Farimani. Scalable transformer for pde surrogate modeling. In Advances in Neural Information Processing Systems, 2024.
[12] Seungjun Lee. Mesh-independent operator learning for partial differential equations. In ICML 2022 2nd Al for Science Workshop, 2022.
[13] Shuhao Cao. Choose transformer: Fourier galerkin. In Advances in Neural Information Processing Systems, 2021.
[14] Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. Transolver: A fast transformer solver for pdes general geometries. In International Conference on Machine Learning, 2024.
[15] Weinan E and Bing Yu. The deep ritz method: a deep learning-bar numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 6(1):1-12, 2018.
[16] Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. Factorized fourier neural operators. In The Eleventh International Conference on Learning Representations, 2023.
[17] Md Ashiqur Rahman, Zachary E. Ross, and Kamyar Arizzadenesheli. U-no: U-shaped neural operators. Transactions Machine Learning Research, 2023.

---

[18] Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M Benson. U-fno-am an enhanced fourier neural operator- based deep-learning model for multiphase flow. Advances in Water Resources, 163:104180 2022.
[19] Oded Ovadia, Adar Kahana, Panos Stinis, Eli Turkel, Dan Givoli, and George Em Karniadakis. Vito: Vision transformerroperrtor. Computer Methods in Applied Mechanics and Engineering, 171:11, 2024.
[20] Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
[21] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer Assisted Interven- lion, pages 234-241, 2015.
[22] Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, and Jun Zhu. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning, 2023.
[23] Zipeng Xiao, Zhongkai Hao, Bokai Lin, Zhijie Deng, and Hang Su. Improved operator learning by orthogonal attention. In Forty-firsi International Conference on Machine Learning, 2024.
[24] Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. arXiv preprint arXiv:2001.0444 2020.
2020.
[25] Jerzy Grotowski. Performer. In The Grotowski Sourcebook, pages 376-380. Routledge, 2013.
[26] Jianbo Shi and J. Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):888-900 2000.
[27] Jean Feydy, Thibault Séjourné, François- Franvois-Xavier Xavier Vialard, Shun-ichi Amari, Alain Trouvé, and Gabriel Peyré. Interpolating between optimal transport and mmd using sinkhorn divergences. In Proceedings of the 22nd International Conference Artificial Intelligence and Statistics (AISTATS), pages 2681-2690 PMLR, April 2019.
[28] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, 2017.
[29] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 652-660,, 2017.
[30] Hongyang Gao and Shuiwang Ji. Graph u-nets. In International Conference Machine Learning, 2019.
[31] Tobias Pfaff, Meire Fortunato, Alvaro Sannhez- and Peter W Battaglia. Learning mesh- based simulation with graph networks. In International Conference on Learning Representations, 2021.
[32] Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, and Jun Zhu. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning, 2023.
[33] Nobuyuki Umetani and Bernd Bickel. Learning three-dimensional flow for interactive aerody- namic design. ACM Transactions on Graphics (TOG), 37(4):1-10, 2018.
[34] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-iia -rcch 3d model repository. arXiv preprint arXiv: 12.00.302222... 2015.
[35] Florent Bonnet, Jocelyn Mazari, Paola Cinnella, and Patrick Gallinari. Airfrans: High fi- delity computational fluid dynamics dataset for approximating reynolds-averaged navier-stokes solutions. In Advances in Neural Information Processing Systems, 2022.
12

---

[36] Gengxiang Chen, Xu Liu, Qinglu Meng, Lu Chen, Changqing Liu, and Yingguang Li. Learning neural operators on rimannnian manifolds. arXiv preprint arXiv:2302.08166 2023.
[37] Qilong Ma, Haixu Wu, Lanxiang Xing, Shangchen Miao, and Mingsheng Long, Deeplag: Discovering deep lagrangian dynamics for intuitive fluid prediction. In Advances in Neural Information Processing Systems, 2024.
[38] Lu Lu, Xuhui Meng, Shengze Cai, Zhiping Mao, Somdatta Goswami, Zhongqiang Zhang, and George Em Karniadakis. A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data. Computer Methods in Applied Mechanics and Engineering, 393:1 14778, 2022.
[39] Ulrike Von Luxburg. A tutorial on spectral clustering, Statistics and computing, 17(4):395-416. 2007.
[40] O.C. Zienkiewicz, R.I. Taylor, and P. Nithiarasu. The Finite Element Method for Fluid Dynamics (Seventh Edition). Butterworth-Heinemamn 2014.
[41] Jeffrey Van Doormaal and George D Raithby. Enhancements of the simple method for predicting incompressible fluid flows. Numerical Heat Transfer, 7(2):147-163, 1984.
[42] Florian Menter, Martin Kuntz, Robin Langtry, et al. Ten years of industrial experience with the sst turbulence model. Tarbulence, Heat and Mass Transfer, 4(1):625-632, 2003.
[43] Wille R. Karman vortex streets. Advances in Applied Mechanics, 6:273-287, 1960.
[44] Barnes W McCormick. Aerodynamics, Aeronautics, and Flight Mechanics. John Wiley & Sons, 1994.
[45] Charles Spearman. The proof and measurement of association between two things. The American Journal of Psychology, 100(3/))4411-71, 1987.
[46] Gaurav Gupta, Xiongye Xiao, and Paul Bogdan. Multiwavelet-basee based operator learning for differential equations. Advances in neural information processing systems, 34:24048-24062, 2021.
[47] Haixu Wu, Tengge Hu, Huakun Luo, Jianmin Wang, and Mingsheng Long. Solving high- dimensional pdes with latent spectral models. In International Conference on Machine Learning, 2023.
13

---

A Implementation Details of Spectral Clustering
The process of spectral clustering 39 can be summarized in three main stages (clusters k):
Step 1: Constructing the Affinity Matrix. Given the input data points (œ1, 22, ,,,..., ,,,), the algorithm begins by treating all points nodes in graph. The similarity between these nodes is then quantified and stored in an affinity matrix W by K-Nearest Neighbors (KNN) method. The process follows:
Ir - x
exp
if x, E KNN((,) or Ij E (NN(x;)
Wij Wji
2
0,
otherwise
Step 2: Constructing the Laplacian Matrix. We use the normalized Laplacian matrix, defined as follows:
= W D 1/2
where 7 is the identity matrix, W is the affinity matrix, and D d. E5= Wajj-
Step 3: Computation of Eigenvectors and Clustering Completion. First, compute the first lt eigenvectors (u,,... Uz) of the normalized Laplacian matrix L, forming the eigenvector matrix U E 12*k Second, denote U (y1,-.. .Yn) where each y; E corresponds to the th row of U. Third, cluster the points y; (i 1, ,n) in R with the k-means algorithm into clusters C,,'.,
We employ standard SpectralClustering function from the scikit-leam library, configured follows: Seeeraalcuuurrringnnouuuseeessss
random_state=47
B Visualization of Template Shapes, Boundary Clustering and Spiderweb Tokenization
In this section, present visualizations of the template shapes of Shape-Net Car and AirRANS, together with examples of surface clustering and spiderweb tokenization.
Random Shape
Random Shape 2
Random Shape 3
Random Shape 4
Random Shape
Random Shape
Random Shape
Random Shape 8
Figure 7: Eight randomly selected shapes from the Shape-Net Car dataset.
Figures and 8 show the eight randomly selected shapes from the Shape-Ner Car and AirfRANS. respectively. Figure Figure@showw shows the template shapes of the Shape-Net Car and AirRAANS. Figure I0 shows the clustering of instances of cars in the Shape-Net Car, with individual clustering (i.c., separately clustering each car surface) and our aligned clustering method. The results demonstrated that our proposed aligned clustering can cluster different instances of cars with aligned clusters including the cluster indexes highlighted by different colors.
Figure II shows the point sets of different spiderweb tokens for an instance of a car in the Shape-Net Car dataset. Each row Figure shows the tokens corresponding to the same cluster car surface, with increasing SDF values to the domain boundary, i.e., the car surface, from left to right. We can observe that the different tokens contain point sets with geometric structures resembling the shape of segments of car surface.
14

---

Random Shape
Random Shape
Random Shape
Random Shape
Shape
Shape
Shape
Random Shape
Figure 8: Eight randomly selected shapes from the AirfRANS dataset.
Figure 9: Template shapes of the Shape-Net Car dataset (left) and AiríRANS dataset (right).
Shape1
Individual Clustering
Aligned Clustering
Shape2
Individual Clustering
Aligned Clustering
Shape3
Individual Clustering
Aligned Clustering
Shape1
Individual Clustering Aligned Clustering
Shupe5
Individual Clustering
Aligned Clustering
Shape6
Individual Chastering
Aligned Clustering
Shape7
Individual Clustering Aligned Clustering
Shape8
Individual Clustering
Aligned Clustering
Figure Clustering car surface (i.e., the PDE domain boundary) by conducting spectral clustering on separate cars (denoted as individual clustering), and our proposed aligned clustering.
Figures and 14 show an example of spiderweb tokenization of an instance of the AirfRANS and the Bounded Navier- Stokes dataset, respectively. Different colors highlight different tokens, which
15

---

Chene
Ti
Ta
Clomee 3
Ti
75
75
75
Tex
Chusee
T5
Chooee
Fn
Ta
Tee
75r
%
TLA
Claster
Tu
Te To 71
7/
Te
T1. 7011 T1.11
7E
Figure 11: Point sets of different spiderweb tokens an instance of car. The p-th row visualizes the point sets belonging to tokens Tp.q. 9 1, my for the p-th (p 1, mp) cluster of the car surface.
Ti
Ta
Tat
7
Tax
T
TR
Ta T
7
T6
T
T1
Tas
Tas
Figure 12: An example spiderweb tokenization for an airfoil with different tokens highlighted by different colors.
well split the point sets surrounding the airfoil into sub-regions. As presented in Section 3.1.2, the determination of width, i.e., [dg-1, dg] of each token, to ensure that the regions with [dg- dg] for g 1, ... m. contain the same number of points, thus adaptive to the different density of point clouds. For comparison, Figure 13 shows representative patches learned by Transolver on the AirfRANS dataset.
16

---

1.0
0.8
0.6
0.2
0.0
Figure 13: Representative patches learned by Transolver on the AirfRANS dataset. Colors indicate the magnitude of the weights of patches.
10
18
17
15
14
12
11
0.2
0.2
0.0
0.0
Figure 14: Spiderweb tokenization for Bounded Navier-Stokes dataset highlighted different colors.
Left: mg = 2, m. 5; Right: mB m, 5.
C Detailed Descriptions of Five Datasets and More Implementation Details of Experiments
This dataset designed simulate the behavior of traveling speed of 72 km/h, using realistic physical parameters and applying the Reynolds-averaged Navier-Stokes (RANS) equations for airflow modeling. The input models are derived from the "car" category in ShapeNet [34), with modifications to remove elements like side mirrors, spoilers, and tires, focusing more on the vehiele's aerodynamic properties. High-resolution spatial grids used solve the Navier-Stokes equations, simulating airflow for a 10-second duration. The time-averaged velocity and pressure fields are obtained by averaging the results from the last 4 seconds of the simulation. Each simulation involves degrees of freedom ranging from 600k to 700k, and each run takes around 50 minutes complete. A finite element Navier-Stokes solver resolves the fluid dynamics, using k-epsilon turbulence model and SUPG stabilization technique (40). This setup accurately represents the flow around the vehicle, capturing complex phenomena such boundary layers and vortex shedding at the surface.
Shape-Net Car dataset.
17

---

For the Shape-Net Net Car dataset, the space is discretized into 32186 mesh points. Overall, total of 889 samples with different car shapes are generated to simulate. 789 samples are used for training, and the other 100 samples are for testing, which follows that of the Transolver [14]. The model is trained to predict the velocity and pressure value for each point, then the drag coefficient can be calculated based on these estimated physical fields.
For the Shape-Net Car dataset, the input dimension of our model is 32186 X 7, where 32186 represents the number of discretized points in the PDE domain, and 7 corresponds to the point coordinates, SDF values, normal vectors of boundary points and direction vectors of interior points. The output of our model is 32186 x 4, where 32186 represents the number of discretized points in the PDE domain, and 4 corresponds the wind field of interior points and the pressure of boundary points (car surface). The drag coefficient can be calculated based on these predicted physies quantities.
AirfRNN dataset. AirfRANS R5 contains high-fidelity simulation data for RANS equations on airfoils that are the 4 and 5-digit series of the National Advisory Committee for Aeronautics. Each case is discretized into 32,000 mesh points. By changing the airfoil shape, Reynolds number, and angle of attack, AirRANN provides 1000 samples, where 720 samples are used for training, 80 samples are used for validation, and 200 for the test set. The model is trained predict the air velocity, pressure, and viscosity for each point, and calculate the lift coefficient. AirIRANS includes high-fidelity simulation data for RANS equations applied to both the NACA 4-digit and 5-digit series airfoils. Specifically, 1000 simulations were run, each defined by the airfoil, Reynolds number, and angle of attack. The goal of the simulations is to reflect realistic conditions, with the Mach number capped at 0.3 and the Reynolds number ranging from 2 million to approximately 6 million. The angle of attack varies from -5" to 15 The simulations were performed using the steady-stat RANS solver simple FOAM with the SIMPLEC algorithm [41]. combined with the k-w SST turbulence model (42, until the lift and drag coefficients converged.
For the AirRANN dataset, the input dimension of our model is 32000 X 7, where 7 corresponds to the point coordinates, inlet velocity, SDF values, normal vectors of boundary points, and direction vectors of interior points. The output of our model is in size of 32000 x 4. where 32000 represents the number of discretized points in the PDE domain, and corresponds to the wind field of interior points, the pressure of boundary points (airfoil surface), and turbulent viscosity of interior points. We aim to estimate the lift coefficient of airfoils, as well as the pressure on both interior and boundary points.
Blood Flow dataset. Blood Flow (36] contains the simulation data of the Navier-Stokes equations for the blood flow dynamics in the human thoracic aorta. In this dataset, the computational domains are the same, i.e., the human thoracic aorta, but the pressure and velocity at the inlet/outlets are given 500 different samples as boundary conditions. The spatial domain is represented by a tetrahedral mesh with 1656 nodes and the temporal domain discrete with 121 temporal time points. 400 samples are used as training data, 50 samples are used as validation data and the remaining 50 as test data, The model is trained to output the velocity field of the blood flow. This dataset contains simulation data for the Navier-Stokes equations modeling blood flow dynamics in the human thoracic aorta. The aorta has one inlet (ascending aorta) and five outlets (descending aorta, left/right subclavian arteries, and left/right common carotid arteries), with velocity boundary conditions at the inlet and pressure boundary conditions the outlets. The dataset describes the time-varying veloeity and pressure over one cardiac cycle s). Blood is modeled as a homogeneous Newtonian fluid with density of 1060 kg/m 3 and viscosity of 0.0035 N-s/m². assuming laminar flow in the aorta and rigid vessel walls with no-slip conditions. A total of 500 velocity/pressure curves are used as inputs, and the velocity field is simulated as the output using COMSOL Multiphysics (36).
For the Blood Flow dataset, we aim to predict the velocity field the interior points of the aorta using pressure values at the inlets and outlets of the aorta. The geometry of the aorta remains consistent across samples, eliminating the need for aligned boundary clustering for different instances of the object surface. For the Blood Flow dataset, the spatial domain is represented by a tetrahedral mesh with 1,656 nodes, and the velocity field is characterized by dimension of 1656 1 x 3. Pressure values at the two inlets and four outlets are recorded across 121 time steps, with a variable dimension of x 121 6. The first two dimensions of the above two variables are replicated to match each other, and they are concatenated along the third dimension, yielding a variable dimension of 1656 x 121 x 9. This is then resized to 1656 x 1089 and used as input to our model. The output of the model has a
18

---

dimension of 1656 x 363, and it is resized to 1656 x 121 x 3, representing the predicted velocity field at each point across the 121 time steps.
Bounded Navier-Stokes dataset. The Bounded Navier-Stokes dataset simulates incompressible dye flow governed by the Navier-Stokes equations, with Reynolds number of 256. At this regime, flow instability leads to periodic vortex shedding and the formation of a Kármán vortex street [43] behind cylindrical obstacles. The presence of multiple cylinders and downstream obstructions induces complex interactions that challenge predictive models. To generate the data, [37 run simulations over 10y steps with fixed cylinder positions but varying initial conditions. From each trajectory. multiple frames are extracted and randomly partitioned into training and test sets. The numerical solver employs finite difference scheme (MAC method) with CIP-based advection, enabling high- resolution flow capture while minimizing numerical dissipation.
The model predicts the physical field at the 10th time step based the initial conditions. total of 1,000 training sequences, 200 validation sequences and 200 test sequences are used at a spatial resolution of 64 x 64. Despite the presence of separate boundaries, the signed distance function (SDF) at each point is uniquely defined as the minimum distance to all boundary components. Consequently, our SDF-based spiderweb tokenization produces non-overlapping partitions, even in such complex domains.
Darey Flow dataset. The Darcy Flow equation is second elliptic PDE, which is defined as
V (u(x)) = f(x), u(x) 0,
E (0,1),
(6)
E 1)
where (x)) is the forcing function, a(x) is the diffusion coefficient, and u(xx) is the density of fluid. We use the same parameter settings for the Darcy Flow equation as in FNO B. The diffusion coefficient axx) is generated from V,,((0, (-A + (-911) with zero Neumann boundary conditions on the Laplacian operator A. The Vy is the point-wise pusw-forward operator that takes the value of 12 if x > 0, and 3 elsewhere. We learn the mapping from a(x) to the steady state u(z), fixing the foreing term f(x) 1. We use 1000, 200, and 200 pairs of (xx and u(x) in the train, validation, and test sets, respectively. The Darey Flow equation dataset is set to four resolutions, i.e., 85, where 8 represents that both the a(x) and u(x)) are discretized into s x 8 grid.
For the Darcy Flow Dataset, the coefficient a(x) results in disconnected multi-regions. In such a dataset, we can take the spatial outer rectangle boundary as the boundary to compute clustering and SDF for tokenization.
Table shows more implementation details for the network structure and network training.
Table 9: Settings of hyper-parameters in the SpiderSolver network and SpiderSolver training.
Dataset
m.
mB
L
h
C'
Loss
Epochs
LR
Optimizer Batch Size
Shape-Net Car
5
4
8
8
256 32
Relative L2
200
0.001
Adam
1
AirfRANS
8
256
32
Relative L2
400
0.001
Adam
Blood Flow
5
6
8
8
512
64
Relative L2
500
0.001
Adam
10
Bounded Navier-Stokes
8
256
32
Relative L2
200
0.0005
Adam
10
Darcy Flow
8
8
8
8
256 32 Relative L2
1000
0.001
Adam 4
Elasticity dataset. The Elasticity dataset consists of simulations on unit cells with arbitrary voids at the center, where the bottom edge is clamped and tension is applied on the top. Each case takes as input a tensor of shape 972 x 2 representing the 2D positions of discretized points, and outputs the corresponding stress field in the shape of 972 x 1. A total of 1000 training samples and 200 test samples are provided, obtained using a finite element solver with about 100 quadratic quadrilateral elements. The inputs are represented as point clouds of size around 1000, and the outputs correspond to the stress fields. This dataset features an unstructured input format, which is designed to evaluate the capability of models handle irregular geometries.
The Plasticity dataset is constructed from plastic forging problems, where a block of material is impacted by a rigid die with random shapes sampled from spline interpolations. The simulations are performed on a 101 x 31 structured mesh with time steps, resulting in 900 training samples and 80 test samples. Each case takes input the die shape discretized into structured mesh
Plasticity dataset.
19

---

of size 101 x 31, and outputs the deformation of each mesh point over 20 time steps in tensor of shape 20 x 101 31 4, containing the deformation in four directions. The outputs correspond to the deformation fields induced by time-dependent die motions. This dataset emphasizes temporal dynamics and structured mesh inputs, providing a challenging benchmark for models that combine spatial and temporal learning.
D Shape Distribution of Shape-Net Car and Error Analysis across Shape Distance
In this section, we quantify shape variation using the distance between each car and the template shape, and analyze the distribution of car geometries. We further visualize how prediction error varies with shape differences, but observe no clear correlation between shape distance and model accuracy.
Distance from each car shape the template shape. We align each shape to the template shape and compute the average point-wise Euclidean distance from all 789 training car surfaces to the template shape, summarized in Table TO
Table 10: Distance to template shape
Cases
Distance (meter)
Min
0.03254
Max
0.6296
Mean
0.1802
Standard Deviation
0.09741
Top 200 closest distances
[0.03254, 0.1031]
Top 100 farthest distances
[0.3005, 0.6296]
The template shape serves as an intermediate reference for aligning and clustering different geometries. Figure [5 presents the template shape, along with the four nearest and four farthest shapes to the template shape, respectively. The distance to the template is computed the average point-wise Euclidean distance following optimal transport (OT) alignment. As shown in Figure 1, this distance metric captures shape variation across the Shape-Net Car dataset.
Closest shape (0.032)
Closest shape (0.033)
Closest shape (0.038)
Closest shape (0.039)
Template shape
Farthest shape (0.52) Farthest shape (0.56)
Farthesi shape 3 (0.59) Farthest shape (0.63)
Figure 15: Template shape and closest and farthest four shapes the template shape, with distances (meters) in parentheses.
Shape distribution of cars on Shape-Net Car dataset. To characterize the shape distribution, we visualize the distances from training shapes to the template shape using a histogram and Q-Q plot in Figure [6] The histogram reveals right-skewed distribution with a gradually decaying tail, while the Q-Q plot indicates significant deviation from normality in the upper quantiles, suggesting heavier- man-Gaussian tails. Note: A Q-Q plot assesses whether dataset follows reference distribution by comparing their quantiles. the context of heavy-tailed detection, a pronounced upward deviation of the upper quantiles from the reference line indicates that the data exhibits heavier tails than the reference (e.g.. normal) distribution.
Analyzing error across shape distance of cars. We further analyze how prediction error varies with the distance to the template shape on the test set, as shown in Figure 17 (ours) and Figure (Transolver). For the prediction of our model, the two shapes farthest from the template 0.5 m) do
20

---

Q Q Plat against Normal Distribution
Sample Quantiles
Frequency
00-
-02
Distance to template shape (meter)
Theoretical Quantiles
Figure 16: The histogram and Q-Q plot of different distances to template shape of Shape-Net Car Dataset.
not yield the highest prediction error.. No clear correlation is observed between shape distance and prediction accuracy.
Rebovve E5 ever velunee
8.00
0.0
(335
82
6.1
Distance from template shape (meter)
Distance from template shape (meter)
Distance template shape Imeter)
Figure 17: Our model's 's testing error with different car distance to template shape on Shape-Net Car Dataset.
0.5
Detattve arroe 01
of
308
03
63
02
Distance tamplate shape (meter)
Distance template shape (meter)
Distance from template (meter)
Figure 18: Transolver's testing error with different car distance template shape on Shape-Net Car Dataset.
E Inference Time and Evaluation Metrics
Inference time. The inference time of our model comprises the spiderweb tokenization (includes optimal transport-based matching) and the network forward pass. Spectral clustering is only per- formed once for each shape during training; during inference, cluster labels propagated from the template shape to the test shape via optimal transport-basee boundary points correspondences. For the Shape-Net Car dataset, spectral clustering is performed on 3,586 surface points using only the top m& eigenvectors of the Laplacian matrix (e.g., 6 eigenvectors for my = -6), which keeps the computational cost manageable. We report the model inference time in Table [1] Note that the prediction commonly takes around 50 minutes, which required by the traditional k-epsilon turbulence simulations.
Comparison of models with and without Fine-AT. We also compare the inference time of the model with and without the Aine-AT on the Shape-Net Car dataset, shown in Table 12
21

---

Table 11: Inference time and number of parameters on the Shape-Net Car dataset.
Different terms
SpiderSolver (Ours)
Transolver
Time of spectral clustering (only model training)
0.26s
N/A
Number of surface points of car
3586
3586
Time of point matching via optimal transport
0.012s
N/A
Time of interior domain partition before model forward
0.0032s
N/A
Time of model forward
0.043s
0.022s
Total inference time
0.058s
0.022s
Number of parameters
4050340
3860804
Table 12: Comparison of Inference time of Fine-AT on the Shape-Net Car dataset.
Setting
Model Forward time (s)
Total Inference time (s)
SpiderSolver (Ours) w/o Fine-AT
0.032
0.047
SpiderSolver (Ours) w/ Fine-AT
0.043
0.058
Since our evaluation is based on practical tasks of different datasets, we include several design- oriented metrics for physics fields, which will be introduced as follows.
Relative L2 error for physics fields. The relative L2 error L(u, ú) between the true physics field and the model's predicted field û is calculated follows:
||
L(u, û)
|
Drag and lift coefficients. For the Shape-Net Car and AirfRANS datasets, drag and lift coefficients are computed from the estimated physics fields. For unit-density fluids, the coefficients are given by:
2 A (1 p(w) (π(w)) i(w)) dw [ T(w) ((w) dw)
C
Jon.
8
where is the inlet flow velocity, A is the reference area, and ön is the object surface. The pressure p. the outward unit normal n. the flow direction i, and the wall shear stress The shear stress 7 is typically much smaller than the pressure term and can be approximated by the air velocity near the surface [44). We use the setting of drag and lift coefficient from Transolver (14)
defined as usual.
Spearman's rank correlation for drag and lift coefficients. Given K test samples, let the true drag or lift coefficients be = {D' ..... D and the predicted coefficients be D = {D) ..... D*}.
The Spearman rank correlation is computed by ranking both sets of coefficients and then calculating
the Pearson correlation between the ranks 14:
cov (R(D), R(D))
R R())
where R is the ranking function, cov denotes covariance of the ranks, and a is the standard deviation. Higher p indicates better alignment between the predicted and true coefficients (45).
F Results on Bounded Navier-Stokes, Darcy Flow, elasticity and plasticity datasets
Tables 13 and show the results on Bounded Navier-Stoff and Darcy Flow, respectively. Table shows the results on Elasticity and Plasticity datasets.
22

---

Table 13: The results on the Bounded Navier-Stokes dataset.
Methods
Relative L2 error !
FNO 13
0.0472
GNOT
32
0,0589
Galerkin
[6]
0,4908
Factformer
0.0452
Transolver 14
0,0555
DeepLag [37
0.0382
SpiderSolver
0.0376
Table 14:
The results on the Darey Flow dataset.
Methods
Relative L2 error 4
FNO 3
0.0108
WMT |46
0,0082
GEO-FNO 66
0.0108
F-FNO 16
0,0077
Galerkin
13
0.0084
LSM
47
0.0065
Transolver 14
0.0068
SpiderSolver
0.0064
G More Visualization Results on Shape-Net Car and Blood Flow
Figure 19 visualizes the Shape-Net Car estimation results by Transolver and our model. The visual results demonstrate that our approach achieved less error in the estimation of physical fields, especially the press value the car surfaces.
Figure 20 visualizes the blood flow estimation results by different compared methods. The estimation absolution error, i.e., the point-wise L2 norm of the difference between ground truth and prediction values of different models, highlighted by heatmaps. The visual results demonstrate that our approach achieved less error in the estimation of physical fields.
Ground Truth (Dry codfficient. (12561)
Translver
Ours (Diag coefficiet: 0.5539)
Thansolver Estimation maps
Ours Estimation enor maps
(Diag coefficien: 0.2566)
Grond Truth (Dng coefficint:: 0,2500)
Transolver (Drag (.2627)
Thasskvar Estimetion entrmeps
(Drag codfident:
Estimetion enoor meps
Ground Truhh
Transolver Estimation ener maps
(Drag 0,2404)
(Drog codficient 0,2406)
(Drag coefficient 0,2427)
Estination enur
Figure 19: Comparisons of the Shape-Net Car estimation by Transolver and model, with the estimation error maps, i.c., point-wise L2 norm of the difference between ground truth and prediction.
23

---

Table 15: The results (Relative Le error) on the Elasticity and Plasticity datasets.
Methods
Elasticity +
Plasticity +
LSM
0.0218
0.0025
GALERKIN
0.0240
0.0120
OFORMER
0.0183
0,0017
GNOT
0.0086
0.0336
ONO
0.0118
0.0048
Transolver
0.0064
0.0012
SpiderSolver
0.0061
0.0011
14
10
Ground Truth
08
04
02
0.0
1.4
100
GNOT
0.6
04
02
000
-D((00111)
12x(3/122))
GUIS
Estimation
maps of GNOT
0.00
0.002
024600.1899)
096()11551
0.000
100
08
Transolver
44
0.2
0246)(01000))
r- (.722(000777)
124
0L15
Estimation error
0.00
maps of Transolver
0.00
(:(0L177)
024:02172)
04486243022
4125)
(96)
(2x(31778)
088
Ours
0.0
0722/00416)
(955(60235)
1.21002297)
Estimation error maps of Ours
0012
0
S0.1224)
<- 1560011991
0722)03397)
(.210.1089)
Figure 20: Comparisons of the blood flow estimation by different methods, with the estimation error maps, i.c., wise Le norm of the difference between ground truth and prediction.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See abstract and introduction section for details.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Conclusion section for details.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
25

---

Justification:
No theoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: See the Experiments section and Appendix for details.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
26

---

Answer: [Yes]
Justification:
We will release the codes after possible publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeeee 7/nips.com htc/://bbb//gudddd///eeeeeueisss ssiniiiiccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
See the Experiments section and Appendix for details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: expensive.
The error bars are not reported because it would be too computationally
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
27

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See the Experiments section and Appendix for details.
Guidelines: See section Experiments.
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We have read the code of ethics carefully and ensure there is no violation.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See Conclusion section for details.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
28

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
There are no contents concerning safeguards.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: We were unable to find the license for the datasets used.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
29

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: There are no new assets introduced in the paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: There is no Crowdsourcing and Research with Human Subjects in this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer:[NA]
Justification:
There are no potential risks as far we are concerned.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
30

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
The LLM is used only for writing, editing, formatting purposes.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy fattps://neur// 220 for what should or should not be described.
31


Reviews:
[{"review_id": "fmYphGrolb", "reviewer": "Reviewer_Rhww", "summary": "This paper introduces SpiderSolver, a novel deep learning model designed to solve PDE on domains with complex geometries and irregular discretizations. The key innovation is a geometry-aware transformer architecture that addresses the challenges of applying standard transformers to such problems.\n\nThe technical contributions include:\n1. The proposed method first uses spectral clustering on the boundary points of the domain to create shape-aware partitions. Then, it divides the interior domain into \"spiderweb-like\" patches based on these boundary clusters and the distance of interior points from the boundary. This tokenization is designed to be more physically intuitive and computationally efficient than methods that treat all mesh points individually.\n\n2. To ensure consistent tokenization across different instances of a similar geometry (e.g., various car shapes), the authors propose an alignment strategy. They use optimal transport to match the boundary points of any given shape to a pre-defined template shape, transferring the clustering from the template to the new instance. \n\n3. SpiderSolver transformer employs a bi-level attention system to understand the coarse-grained and fine-grained features. \n\nThe authors evaluate SpiderSolver on five diverse datasets, including simulations based on Shape-Net Car/AirfRANS, blood flow in the human aorta, and canonical Navier-Stokes and Darcy flow problems. The results demonstrate that SpiderSolver consistently achieves SOTA performance, outperforming previous neural operators and transformer-based PDE solvers, particularly in its generalization to OOD cases.", "questions": "Please address the weakness points I mentioned.", "limitations": "Please address the weakness points I mentioned.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "S1. The core idea of \"spiderweb tokenization\" is a significant and intuitive contribution. \nS2. The dual-level attention mechanism is an intuitive yet effective way to mitigate the quadratic complexity of standard transformers. \nS3. The use of optimal transport for aligning boundary clustering is a sophisticated solution to handle variations in shape within a class of objects.\n\nW1. The authors acknowledge that the current formulation of SpiderSolver relies on a well-defined, closed boundary. This limits its applicability to problems with open-boundary domains or domains with multiple, disconnected internal boundaries that are not part of a single, continuous surface. While the Bounded Navier-Stokes dataset has multiple obstacles, the tokenization still relies on the minimum distance to any boundary component, which may not be optimal for capturing interactions between the obstacles themselves. \n\nW2. The aligned clustering method requires a \"template shape\" for each class of geometry. The paper does not fully detail how this template is chosen or how sensitive the model's performance is to this choice. An arbitrary or poorly chosen template could potentially lead to suboptimal tokenization for the entire dataset.\n\nW3. While the paper notes that spectral clustering is computationally manageable for the datasets used (e.g., around 3,600 surface points for the car), this treatment could become a bottleneck for problems with extremely high-resolution boundary meshes containing millions of points.", "was_revised": true, "final_justification": "Many thanks for providing further dicussions and results. They fully addressed my concerns. I will keep my score.", "rebuttal": "### **Q1: Reduced suitability for open or multi-boundary domains, not optimal for obstacle interaction modeling.**\n\nWe first clarify the applicability of SpiderSolver to open and multi-boundary domains, and then explain how obstacle interactions are already modeled within our framework. Finally, we conducted ablation studies to support our claims.\n\n***(1) Clarification on the applicability of SpiderSolver to boundary conditions.***\nWe acknowledge that SpiderSolver is currently not applicable to PDEs with open-boundary domains, as stated in 'Limitations and future work'  in manuscript.\nHowever, it does support domains with multiple disconnected internal boundaries, such as those present in the Bounded Navier-Stokes dataset.\n\n***(2) Obstacle Interaction Analysis.***\nSpiderSolver performs spectral clustering on the surfaces of all obstacles, producing $m_B$ boundary clusters. The interior domain is then partitioned into $m_B m_I$ subregions, resulting in a total of $m_B m_I  + m_B$ tokens.\n**Token-level attention captures obstacle interactions:** Coarse-grained Attention in SpiderSolver naturally captures interactions between the obstacles, as the $m_B m_I + m_B$ tokens attend to each other directly.\n**Shape-aware partitions reflect obstacle geometry:**  As shown in Figure 14 (page 15), the interior partitions closely align with the geometry of the obstacles. These shape-aware partitions ensure that the learned attention maps inherently encode the mutual influence among different obstacles.\n\n***(3) Ablation of merging obstacle surface tokens.***\nWe conduct an ablation study in which the $m_B$ tokens corresponding to obstacle surfaces are merged into a single token, resulting in a total of $m_B m_I  + 1$ tokens. The results are reported in Table R1-1. The experimental results indicate that explicitly modeling interactions between obstacles leads to improved model performance.\n\nTable R1-1. Effect of obstacle surface token merging on prediction accuracy.\n\n| Methods  | Relative $L_2$ error ↓ |\n|----|---|\n| SpiderSolver with merged obstacle surface tokens  | 0.0432 | \n| SpiderSolver (Ours)  | 0.0376 | \n\n###  **Q2: Template selection is unclear, with potential sensitivity and suboptimal tokenization arising from poor template choices.**\n\nWe first clarify the selection of the reference shape and the construction of the template shape, highlighting the distinction between the two.\nWe then performed an ablation study with different reference shapes.\n\n***Reference shape and template shape.***\nThe reference shape is randomly selected from the training data, and the template shape is then obtained by averaging the points of all training shapes.\nTo enable meaningful pointwise averaging, all shapes are first aligned to the reference shape using optimal transport.  The reference shape serves only to provide a consistent indexing, and it does not affect the geometry of the resulting template shape.\n\n***Ablation on reference shape selection.***\nAs shown in  Table R1-2, our model exhibits strong robustness to the choice of reference shape on the Shape-Net Car, consistently outperforming Transolver across different reference shapes in Table R1-2.\n\nTable R1-2.  Ablation study on reference shape selection for Shape-Net Car.\n\n| Methods | Vol ↓ | Surf ↓ | $ C_D $ ↓ | $ \\rho_D $ ↑ |\n|----|---|--|----|---|\n| Reference shape 1  | 0.0215  | 0.0730  |  0.0095  | 0.9934   |\n| Reference shape 2  |  0.0212 | 0.0721 | 0.0103 |  0.9935 |\n| Reference shape 3  |0.0215   | 0.0725| 0.0105 | 0.9921 |\n| Reference shape 4  |  0.0210 | 0.0727 | 0.0103 | 0.9934 |\n| Results in manuscript  |   0.0210  | 0.0738 | 0.0100  | 0.9928   |\n| **Mean** |  0.0212  |  0.0728  |  0.0101  | 0.9930  |\n| **Standard Deviation** |  0.0002  |  0.0006  |  0.0003 | 0.0005  |\n| Transolver |  0.0228  |  0.0793  | 0.0129 | 0.9916 |\n\n***Measuring clustering consistency under different reference shapes.***\nAs shown in Table R1-3,\nDice and IoU are used to evaluate the consistency of spectral clustering results of five template shapes based on five different reference shapes in Table R1-2. Dice and IoU are computed based on the intersection and union of the five clusterings.\n\nTable R1-3. Measuring clustering consistency under five reference shapes.\n| Case|  Dice ↑ | IoU ↑ |\n|:---:|:---:|:---:|\n| Five clusterings | 0.9975  |0.9951 |\n\n###  **Q3: Computational cost of spectral clustering on high-resolution meshes.**\n\nWe first describe the current strategies employed in our model to reduce the computational cost of spectral clustering. Secondly, we add the model training with subsampling-guided clustering.\nFinally, we outline our planned solution for efficiently handling boundary with millions points in future extensions.\n\n***(1) Reducing spectral clustering cost in our current model.***\nOur model incorporates two strategies to reduce spectral clustering cost: template-based transfer to avoid per-instance clustering, and a sparse Laplacian decomposition using k-nearest neighbor affinity.\n\n**Single clustering via template transfer:** During training, spectral clustering is performed only once on the template shape (i.e., average shape of aligned geometric category), rather than on every individual instance. The clustering results are then transferred to other individual shapes using optimal transport. No spectral clustering is needed during inference.\n\n**Efficient spectral decomposition with neighbor-based affinity:**\nWe adopt an efficient spectral clustering method based on a k-nearest neighbor graph, and only the top $m_B$ eigenvectors (typically $m_B=2 \\sim 8$) of the Laplacian matrix are computed.\nTable R1-4 compares our clustering approach with the classical radial basis function kernel \n(RBF-based) method [1], which constructs a fully connected affinity matrix based on pairwise distances.\n\nTable R1-4. Comparison for spectral clustering on a car surface with 6 clusters.\n| Methods | Number of Points | Memory Usage (MB)  ↓| Time (s)  ↓|\n| :---: | :---: | :---: | :---: |\n|Classical RBF| 3586 |551.90 | 3.06 |\n|Our approach| 3586 |138.10 | 0.26 |\n\n**Clustering overhead analysis:** Table R1-5 provides a clearer assessment of the computational overhead.\nThe original car surface contains 3,586 points. Higher-resolution surfaces were generated using the feature-preserving edge-aware resampling (EAR) algorithm, while lower-resolution surfaces were obtained via standard uniform random sampling.\n\nTable R1-5. Computational cost of spectral clustering under different surface resolutions.\n\n| Sampling Rate | Number of Points | Memory Usage (MB)  ↓| Time (s)  ↓|\n| :---: | :---: | :---: | :---: |\n|1/8 | 448 | 132.90 | 0.04 |\n| 1/6 | 597 | 133.80 | 0.05 |\n| 1/4 | 896 | 133.91 | 0.07 |\n| 1/2 | 1793 | 134.57 | 0.15 |\n| 1 | 3586 (Original) | 138.10 | 0.26 |\n| 2 | 7172 | 146.57 | 1.04 |\n| 4 | 14344 | 168.00 | 4.54 |\n| 6 | 21516 | 188.73 | 9.50 |\n| 8 | 28688 | 212.23 | 20.74 |\n| 16 | 57376 | 297.82 | 90.39 |\n| 32 | 114752 | 468.07 | 373.37 |\n\nAs shown in Table R1-5, clustering becomes time-consuming at high resolutions.\nTo address this, we propose a subsampling-based clustering strategy and conduct experiments to verify its effectiveness.\n\n***(2) Speeding up the spectral clustering on larger scale boundary point clouds by  subsampling-based spectral clustering.***\n\nTo efficiently speed up the spectral clustering on large scale boundary point cloud, we can first subsample the original point cloud and compute the spectral clustering on the subsampled boundary point cloud, then upsample the clustering by labeling the remaining points in the original point set  using k-nearest neighbor (KNN) classifier.  To evaluate this idea, we use the edge-aware resampling (EAR) algorithm to generate high-resolution data on Shape-Net Car. Then the\nspectral clustering is performed on the original 3586 points, and the resulting cluster labels are propagated to the high-resolution points via the KNN classifier.\nWe compare the propagated clustering with the clustering of directly performing spectral clustering on the high-resolution point cloud, as shown in Table R1-6.\nThe subsampling-based clustering yields results that are highly consistent with those obtained from direct clustering at high resolution, while substantially reducing the computational time. \nThe total clustering time under the subsampling-based strategy consists of the spectral clustering time on the original 3586 points (0.26s), plus the time required for label propagation via the KNN classifier (Table R1-6).\n\nTable R1-6. Clustering quality under point cloud subsampling at different resolutions.\n\n| Resolutions | Dice ↑ | IoU  ↑ |Time of KNN Classifier (s) ↓|\n| :---: | :---: | :---: | :---: | \n| 7172  |0.9762|0.9521|   0.0054 |   \n| 14344  |0.9678|0.9393| 0.0125 |   \n| 21516  | 0.9621|0.9278| 0.0207 |  \n| 28688  | 0.9613|0.9257| 0.0277|\n| 57376  | 0.9478|0.9013|  0.0575|\n| 114752  | 0.9324|0.8748| 0.1226 |\n\n**Effect on the performance under subsampling-based spectral clustering:**\nWe futher evaluate the effect of subsampling-based spectral clustering on the final performance. On the Shape-Net Car, we perform the subsampling-based spectral clustering on the subsampled car surface of 896 points (i.e., one-fourth of the original 3586 resolution).  We then train the model using this approximate clustering, as shown in Table R1-7. The model exhibits only a slight performance degradation under this setting, while still outperforming Transolver, demonstrating the effectiveness of our subsampling-based clustering strategy.\n\nTable R1-7. Effect of subsampling-based clustering on model performance.\n\n| Methods    | Vol ↓ | Surf ↓ | $ C_D $ ↓ | $ \\rho_D $ ↑ |\n|-----|---|---|--|--|\n| Transolver | 0.0228 | 0.0793  | 0.0129 | 0.9916 |\n| SpiderSolver (original) | 0.0210  | 0.0738 | 0.0100 | 0.9928 |\n| SpiderSolver with subsampling-based clustering | 0.0213  |0.0742 | 0.0107 | 0.9924|\n\n[1] Ng A, Jordan M, Weiss Y. On spectral clustering: Analysis and an algorithm. NeurIPS 2001."}, {"review_id": "7JK2Pvwydp", "reviewer": "Reviewer_urfu", "summary": "This paper introduces SpiderSolver, a geometry-aware Transformer designed to solve partial differential equations (PDEs) in irregular domains. SpiderSolver introduces spiderweb tokenization, which divides the domain into sections based on the shape's boundaries and distances, allowing for effective attention across different scales. When evaluated on five diverse PDE datasets with intricate geometries, SpiderSolver consistently achieves top performance and shows enhanced generalization compared to existing approaches.", "questions": "1. The results reported in Table 3 of the Transolver do not exactly match those in Table 5. Could you please clarify the differences in the evaluation process?\n\n2. Instead of aligning the boundary clustering for different instances of the object, what would happen if we performed boundary clustering independently for each instance? Would this approach improve performance? This could also serve as an interesting ablation study.", "limitations": "yes. The proposed model requires more time. It should be mentioned in the limitations.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strength\n1. Tokenizing irregularly structured domains, such as graph/mesh/point-cloud, is a challenging problem. The approach introduced here is novel to my knowledge and might be useful even outside of the problem of PDE solving.\n\n2. The proposed technique achieves better empirical results compared to the baselines.\n\n## Weaknesses\n\n1. Although the paper is mostly well-written and clear, the paragraph on \"Boundary Spectral Clustering\" is not explained clearly.\n\n2. It is crucial to follow up on all datasets of the major baselines (such as Transolver) for proper comparison. Datasets like Elasticity and Plasticity are not included.\n\n3. The proposed technique is not evaluated for zero-shot generalization on discretization (of the domain), meaning that the evaluation samples contain more or fewer points than the training samples.\n\n4. Table 9 indicates that the proposed model requires almost twice the time compared to Transolver.\n\n5. The results in Table 6 are not interpretable. It would be beneficial to add a single performance metric to clearly demonstrate its dependence on $m_I$ and $m_B$. Additionally, it is generally expected that as $m_I$ and $m_B$ increase, the model's performance will improve. However, in many cases, this is not true, and this crucial issue remains unaddressed.", "was_revised": true, "final_justification": "The authors have addressed my concerns, including rewriting several sections of the paper, adding new ablations, and discussing some important limitations. I have increased the score.", "rebuttal": "### **Q1: Further clarification on \"Boundary Spectral Clustering\"  paragraph.**\n\nWe apologize for that, and we provide additional implementation details here, which will be included in the appendix of the revised manuscript.\nThe underlying process of spectral clustering [1] can be summarized in three main stages (clusters = k):\n\n**Step 1: Constructing the affinity matrix.** Given the input data points $(x_1, x_2, x_3, \\dots, x_n)$, the algorithm begins by treating all points as nodes in a graph. The similarity between these nodes is then quantified and stored in an affinity matrix $W$ by k-nearest neighbor (KNN) method. The process is as follows:\n\n$w_{ij} = w_{ji} = \\exp \\left( - \\|| x_i - x_j \\||^2 / (2 \\sigma^2) \\right) \\text{ if } x_i \\in \\text{KNN}(x_j) \\text{ or } x_j \\in \\text{KNN}(x_i), \\text{ otherwise } 0.$\n\n\n\n**Step 2: Constructing the Laplacian matrix.** We use the normalized Laplacian matrix, defined as follows:\n\n$\nL=I-D^{-1/2}WD^{-1/2},\n$\n\nwhere $I$ is the identity matrix, $W$ is the affinity matrix, and $D=\\text{diag}(d_1,...,d_n)$ , $d_i=\\sum_{j=1}^nw_{i,j}.$\n\n**Step 3: Computation of eigenvectors and clustering completion.** First, compute the first $k$ eigenvectors $(\\mathbf{u}_1, \\cdots, \\mathbf{u}_k)$ of the normalized Laplacian matrix $L$, forming the eigenvector matrix $U \\in \\mathbb{R}^{n \\times k}$. Second, denote $U = ( \\mathbf{y}_1, \\cdots, \\mathbf{y}_n )^{\\top}$, where each $\\mathbf{y}_i \\in \\mathbb{R}^k$ corresponds to the $i$ th row of $U$. Third, cluster the points $\\mathbf{y}_i (i=1, \\cdots, n)$  in $\\mathbb{R}^k$ with the k-means algorithm into clusters $C_1, \\cdots, C_k$. \n\nWe adopt the standard SpectralClustering function from the scikit-learn library:\nSpectralClustering(n\\_clusters=clusters,affinity=’nearest\\_neighbors’,random\\_state=42).\n\n\n### **Q2: Comparison on Elasticity and Plasticity datasets.**\n\nWe extend our evaluation to the Elasticity and Plasticity datasets, as shown in Table R2-1.\nFor the Elasticity, the void regions of the shape are treated as complex boundaries for spectral clustering, and $m_I = m_B = 8$.\nFor the Plasticity, the four edges of the rectangular region are treated as boundaries for spectral clustering, and $m_I = 4, m_B = 16$.\nWe set the number of patches to 64, consistent with the configuration used in Transolver.\nThe results are presented in Table R2-1, and our method outperforms all other models.\n\nTable R2-1. Performance comparison (Relative L2 error) on Elasticity and Plasticity datasets.\n| Model| Elasticity | Plasticity | \n| :---: | :---: | :---: | \n| LSM | 0.0218 | 0.0025 |\n| GALERKIN | 0.0240 | 0.0120 |\n| OFORMER | 0.0183 | 0.0017 |\n|  GNOT| 0.0086  | 0.0336 |\n| ONO |0.0118  | 0.0048 |\n|  Transolver| 0.0064 | 0.0012|\n |  SpiderSolver (Ours)| **0.0061** |  **0.0011** |\n\nOur method has now been evaluated on a total of seven benchmarks. \nAcross all of these, SpiderSolver achieves state-of-the-art performance. \nThis comprehensive evaluation demonstrates the generality and broad applicability of our approach across diverse PDE systems and geometric settings.\n\n### **Q3: Lacks evaluation on generalization to domains with different discretization resolutions.**\n\nWe have conducted new experiments to evaluate performance under different discretization resolutions.\nLower-resolution inputs are obtained by randomly downsampling the original point cloud, while higher-resolution inputs are generated via bilinear interpolation. The results are summarized in Table R2-2.\n\nTable R2-2. Model evaluation (relative $L_2$ error) with different discretization resolutions.\n\n| Sampling Rate | Resolutions |Transolver |  SpiderSolver |\n| :---: |  :---: | :---: | :---: | \n|  1/4|   414 |  0.0562  |  0.0446   | \n|  1/2 |  828  |   0.0489|   0.0390  | \n|  1 |   1656  (Original)  |   0.0438   | 0.0322  | \n|  2 |   3312  |  0.0892  |  0.0790  | \n|  4  |   6624  |  0.1390  |  0.1023  | \n\nSpiderSolver outperforms Transolver under both settings, demonstrating stronger generalization to resolution changes. However, both models exhibit performance limitations when applied to the upsampled high-resolution, indicating that generalization to fine-grained discretization remains a challenging direction for future work.\n\n### **Q4: Proposed model requires almost twice the time compared to Transolver.**\n\nThis limitation will be explicitly noted in the revised manuscript.\nTo demonstrate the broader value of our method, we provide the following two perspectives.\n\n***Inference efficiency and scientific utility.***\nIt is important to contextualize this overhead relative to traditional numerical solvers. For example, a conventional $k$–$\\varepsilon$ turbulence simulation typically takes around 50 minutes for a single prediction of the Shape-Net Car. In contrast, both Transolver and SpiderSolver produce results within tens of milliseconds, achieving a speedup of over three orders of magnitude. From the perspective of accelerating scientific simulations, millisecond-level inference remains highly practical for deployment.\n\n***State-of-the-Art performance on seven PDE datasets.***\nSpiderSolver has been validated across seven diverse benchmarks. Across all cases, it achieves state-of-the-art performance. This broad evaluation underscores the generality and practical value of our method, even when accounting for its modest increase in inference time.\n\n### **Q5: No clear metric shows how  $m_I, m_B$ affect performance; results are inconsistent and hard to interpret.**\n\nWe observe no clear trend in model performance across different values of $m_I$ and $m_B$ in Table 6.\nWe attribute this to the large number of patches used, which results in too few discrete points within each patch, which is insufficient to capture the underlying physical field characteristics.\nTo validate this hypothesis, we conducted additional experiments using fewer patches. \nAs shown in Tables R2-3 and R2-4, our model's performance improves as the number of patches increases.\n\nTable R2-3. Ablation study of SpiderSolver with varying $m_I$ and $m_B$ on AirfRANS.\n\n| $(m_I, m_B)$| Patch Number ($m_Bm_I+ m_B$)  | Vol ↓ |Surf ↓|$C_L$ ↓ |  $\\rho_L$ ↑ | \n|---|---|----|---|---|---|\n|(2,2)|6| 0.0026 | 0.0130 |  0.0885 | 0.9987 | \n|(4,2) |10|   0.0021 | 0.0101 | 0.0818|0.9988 | \n|(2,4) |12 | 0.0019 |0.0064 | 0.0751|0.9989 | \n|(6,2)| 14 | 0.0020  |0.0079 |0.0624 | 0.9990  | \n|(2,6) |18|  0.0019 |0.0075 | 0.0583 | 0.9988 | \n\n\nTable R2-4. Ablation study of SpiderSolver with varying $m_I$ and $m_B$ on Blood Flow.\n\n|$(m_I, m_B)$|Patch Number ($m_Bm_I+ m_B$) | Vol ↓|  \n|-----|---|----|\n|(2,2) | 6 | 0.0372 | \n|(4,2) | 10 |0.0366 |\n| (2,4) | 12  | 0.0359  |   \n| (6,2) | 14 | 0.0352 |  \n|(2,6) | 18 | 0.0350 | \n\n### **Q6: Clarification on the discrepancy between Table 5 and Table 3**\n\nWe summarize the distinct purposes and experimental setups of Table 5 and Table 3. \n\n***Clarification on Table 3 and Table 5 in our manuscript.***\nWe believe the reviewer may be referring to Table 3 and Table 5 in our manuscript, as we did not find any direct connection between Tables 3 and 5 in the original Transolver paper.\n\n***Different experimental settings.***\nTable 3 employs distinct training and test sets tailored for out-of-distribution (OOD) experiments, which differ from the standard in-distribution setup used in Table 5, including extrapolation with respect to Reynolds number and angle of attack. \nThe corresponding experimental setup is detailed in Lines 321–328 on page 8 of our manuscript.\nTable 5 reports the results of our ablation studies (Lines 341-348 on page 9).\n\n### **Q7: Instance-wise boundary clustering may help, consider ablation.**\n\nWe provide a detailed explanation of the limitations of instance-specific boundary clustering, followed by an ablation study evaluating its effect.\n\n***Limitations of instance-specific boundary clustering.***\nAs shown in Figure 4(b) and Figure 10 in Appendix A, performing boundary clustering independently for each instance results in unaligned clustering. That is, corresponding regions across different instances (e.g., the rooftops of different cars) are not guaranteed to be assigned to the same cluster. This lack of alignment leads to inconsistencies in how the interior domain is partitioned into patches, making it difficult for the attention mechanism to capture spatial correspondences across samples. This observation motivates our use of a template shape (Lines 153–165 on page 5).\n\n***Ablation with instance-specific clustering.***\nTo further assess this alternative, we conducted additional experiments where boundary clustering was performed independently for each instance, and all other components of our model remained unchanged, as shown in Tables R2-5 and R2-6.\n\nTable R2-5. Effect of instance-specific clustering on model performance on Shape-Net Car.\n\n| Model| Vol ↓| Surf ↓ | $ C_D $ ↓ | $ \\rho_D $ ↑ |\n|---|--|---|---|--|\n|SpiderSolver (Ours)| 0.0210 | 0.0738 | 0.0100| 0.9928 |\n|Instance-specific clustering | 0.0222 | 0.0751|0.0124 | 0.9911|\n\nTable R2-6. Effect of instance-specific clustering on model performance on AirfRANS.\n\n| Model | Vol ↓ | Surf ↓ | $C_L$  ↓ | $\\rho_L$ ↑ |\n|---|---|---|---|----|\n| SpiderSolver (Ours)|0.0017| 0.0043 | 0.0741| 0.9988|\n| Instance-specific clustering|0.0022|0.0099 |0.1013| 0.9986|\n\nAs shown in Tables R2-5 and R2-6, instance-wise boundary clustering leads to a performance drop. This degradation arises from misaligned clustering results across different instances, which in turn causes geometric inconsistency in the partitioning of interior points within the PDE domain.\n\n&nbsp;\n\n[1] Von Luxburg U. A tutorial on spectral clustering [J]. Statistics and computing, 2007, 17(4): 395-416."}, {"review_id": "6PGZPx9wcf", "reviewer": "Reviewer_hsJq", "summary": "This paper proposes SpiderSolver, a geometry-aware transformer that introduces spiderweb tokenization for handling complex domain geometry and irregularly discretized points. This method partitions the irregular spatial domain into spiderweb-like patches, guided by the domain boundary geometry. SpiderSolver leverages a coarse-grained attention mechanism to capture global interactions across spiderweb tokens and a fine-grained attention mechanism to refine feature interactions between the domain boundary and its neighboring interior points. Experimental results demonstrate that SpiderSolver consistently achieves state-of-the-art performance across different datasets and metrics.", "questions": "In eq. 5, how is the initial point-wise feature $x^0$ computed? Since spiderweb token features $\\hat{f}^{l}$ is shared for each patch,  the initial point-wise feature must show significant difference among points.", "limitations": "The proposed SpiderSolver is applied to PDE with boundary shapes of one type of objects. If the shape of objects vary dramatically, the tokenization transferred from the template may be unreliable. The spectral clustering for a new object with significantly new shape will be time-consuming. The attention transferred is unreliable too.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. SpiderSolver is a geometry-aware Transformer-based PDE solver, which is designed to partition the domain into spiderweb-like patches utilizing physical and geometric knowledge to reduce the computation cost of attention.\n2. SpiderSolver adopts the multi-grained attention mechanism to capture the intricate physical correlation of complex boundaries and interior points. The fine-grained attention can extract the interact features between points of domain boundary and their near points in the interior of domain to model the intricate  influence of boundary shapes on solutions.\n3. Extensive experimental results are provided.\n\nWeaknesses:\n1. Please give a discussion on the robustness of spiderweb tokenization to the noise in boundary point coordinates.\n2. The time used for spectral clustering is not provided.\n3. The spiderweb tokenization generates patches that vary notably in size. Please give a discussion on the effect of variable patch size.\n4. In the experiment of generalization to shape variations of cars, if the shape of cars vary significantly, the patches obtained by spiderweb tokenization will vary in space across different car instances, how does the attention between patches learned by nearest shapes transfer to patches with varying position?  Please give a discussion on this.", "was_revised": true, "final_justification": "My concerns, mostly on the robustness of the proposed method, have been addressed by authors' rebuttal.", "rebuttal": "### **Q1: Robustness of spiderweb tokenization to noisy boundary points.**\nWe first quantify the robustness of the spiderweb tokenization using standard similarity metrics under increasing noise levels. We then provide an analysis explaining why spectral clustering and SDF-based slicing remain structurally stable under such perturbations. Finally, we evaluate the performance of our model when exposed to noisy boundary conditions at the inference phase.\n\n***(1) Robustness of spiderweb tokenization to boundary noise.***\nTo assess the robustness of our spiderweb tokenization to noise on boundary points, we introduce Gaussian noise with zero mean and standard deviation $\\sigma$ to the car surface point cloud, applied independently to each point coordinate. We then compare the resulting partitioning of the PDE domain before and after noise injection using Dice score and Intersection over Union (IoU)，as shown in Table R3-1.\n\nTable R3-1. Robustness of spiderweb tokenization under car boundary perturbation ($\\sigma$ is noise standard deviation).\n| $\\sigma$ | Dice ↑ | IoU ↑ | \n|--|-|-|\n| 0.0001|1.000 | 1.000 |\n| 0.001 | 0.9994 | 0.9987| \n| 0.01|0.9868 | 0.9743 | \n| 0.03 |0.9534 | 0.9135 | \n\n**Spectral clustering preserves structure under noisy boundaries.**\nThe spiderweb tokenization relies on spectral clustering of the car surface. While noise perturbs the surface point cloud locally, it does not significantly alter the global geometry of the car. Spectral clustering uses the eigenvectors of the graph Laplacian to embed the data in a lower-dimensional space, which enhances global structure while suppressing local fluctuations. As a result, the derived tokenization of the PDE domain, remains stable despite surface noise.\n**SDF-based partitioning is robust to local perturbations.**\nThe interior partitioning of the PDE domain is based on signed distance (SDF) values to the car surface. Since the SDF calculation inherently reflects the noisy surface, any local perturbation only causes a consistent increase or decrease in nearby SDF values. Thus, the overall structure of SDF-based slicing remains largely unaffected, further contributing to the robustness of the tokenization.\n\n***(2) Model performance under noisy boundary conditions.***\nWe evaluate the model's robustness by training on clean data and testing on inputs with noisy boundary point clouds, as shown in Table R3-2. \nThe robustness of patch partitioning within the PDE domain under boundary perturbations (Table R3-1) contributes to the overall stability of the model against such noise.\n\nTable R3-2. Model performance under noisy boundary conditions on Shape-Net Car  ($\\sigma$ is noise standard deviation).\n\n| $\\sigma$ | Vol ↓ | Surf ↓ | $ C_D $ ↓ | $ \\rho_D $ ↑ |\n|---|--|-|--|--|\n| 0.0001 |0.0210 |0.0738 | 0.0100 |0.9928 |\n| 0.001 | 0.0217 |0.0739 | 0.0103 | 0.9924 |\n| 0.01 | 0.0230| 0.0790 | 0.0119 | 0.9919 |\n| 0.03 | 0.0310|0.0829 | 0.0135 | 0.9901 |\n\n### **Q2: The time used for spectral clustering is not provided.**\nThe runtime of the spectral clustering is reported in Table 9 on page 20 of our manuscript, \nae well as the time of point matching via optimal transport, time of interior domain partition before model forward, time of model forward.\nWe will refer to Table 9 at more appropriate locations in the main text to prevent them from being overlooked.\n\n### **Q3: Discussion on the effect of variable patch size.**\n\nWe begin by explaining how the patch size variation naturally arises from our design, then discuss the physical motivations behind using non-uniform spatial resolution, and finally present an ablation study comparing against uniform partitioning.\n\n***Origin of variable patch size in spiderweb tokenization.***\nThe variation in patch size is an inherent result of our spiderweb-based tokenization scheme. This strategy allocates finer patches near the object surface (e.g., the car), while assigning coarser patches to regions farther away. Consequently, patches closer to the geometry have smaller spatial areas, reflecting higher local resolution.\n\n***Physical rationale for non-uniform patching.***\nThis non-uniform partitioning is well aligned with the nature of flow fields. Near the object, the flow exhibits rapid variations in velocity and direction. Smaller patches in these regions are better suited for capturing complex local dynamics. In contrast, regions far from the object typically resemble uniform free-stream behavior, where the physics are smoother and do not require fine spatial resolution.\n\n***Ablation with uniform patch size.***\nTo evaluate the impact of patch size variability, we conducted an ablation experiment using uniform spatial partitioning, dividing the PDE domain into equal-sized patches based solely on coordinates. The results, reported in Tables R3-3 and R3-4, demonstrate the effectiveness of our spiderweb-based variable-resolution strategy.\n\nTable R3-3. Effect of uniform patch size on model performance on Shape-Net Car.\n\n| Model | Vol ↓ | Surf ↓ | $ C_D $ ↓ | $ \\rho_D $ ↑ |\n|---|---|-|---|---|\n| SpiderSolver (Ours) | 0.0210  | 0.0738 | 0.0100 | 0.9928 |\n|Transformer with uniform patch size  |  0.0271  | 0.0996 | 0.0192| 0.9699 |\n\nTable R3-4. Effect of uniform patch size on model performance on AirfRANS.\n\n| Model | Vol ↓ | Surf ↓ | $C_L$  ↓ | $\\rho_L$ ↑ |\n|---|---|--|---|---|\n| SpiderSolver (Ours)  |0.0017  | 0.0043 | 0.0741  | 0.9988   |\n| Transformer with uniform patch size   |  0.0048  |  0.0195   |  0.1356  |    0.9963   |\n\n### **Q4: How tokenization and attention transfers reliably under significant car shape variation.**\n\nWe first analyze the results from generalization experiments, then explain how tokenization and attention remain reliable under significant car shape variation from a methodological perspective.\n\n***(1) Support from generalization experiments.***\nAs detailed in lines 329-339 on page 8 and Appendix C on pages 18-19,\nwe compute the average point-wise Euclidean distance from each car shape to the template shape, and then select the 200 nearest shapes for training and the 100 farthest shapes for testing.\nAs shown in Figure 17 (page 19), there is no discernible correlation between shape distance and prediction error across test samples. This lack of performance degradation, even for cars exhibiting substantial geometric differences, demonstrates that our tokenization and attention mechanisms generalize effectively. The learned representations maintain semantic alignment across diverse car shapes, enabling reliable transfer of information between varying geometries.\n\n***(2) Methodological explanation of tokenization and attention transfer.***\n\n**Semantic consistency through distance-based interior partitioning:**\nIn our method, the interior domain is partitioned by leveraging both the spectral clustering of the boundary and the signed distance of each interior point to the boundary surface. This joint strategy ensures that token regions reflect the geometric layout of each shape.\n\n**Alignment across varying shapes via optimal transport:**\nTo account for shape variation, we apply optimal transport (OT) to align boundary points between different car instances. This alignment provides a pointwise correspondence that preserves the semantic structure across shapes. As a result, although patch positions (by our spiderweb tokenization) may shift spatially due to shape differences, the relative semantic meaning of each patch (e.g., “above the car roof” or “near the rear bumper”) remains consistent.\n\n**Robust attention modeling across aligned patches:**\nSince attention in our model is learned over these semantically aligned tokens, the learned dependencies generalize across instances even when geometric layouts vary. This semantic alignment is essential for robust token-level attention and enables effective knowledge transfer across diverse shapes.\n\n### **Q5: Computation of the initial point-wise feature $x^{0}$，it must show significant difference among points.**\n\nThe initial point-wise feature $x^0$ is obtained by passing the model's input through a two-layer MLP. Although the same MLP is applied to all points, the input itself encodes meaningful spatial and physical differences between points, ensuring that the resulting features $\\mathbf{x}^l$  remain distinct across the domain.\nWe summarize below the specific input representations used for different tasks in our experiments, as shown in Table R3-5.\n\nTable R3-5. Specific input representations for different physical modeling tasks.\n\n| Datasets | Input of our model | \n|---|---|\n| Shape-Net Car | Point cloud, SDF, boundary normals, interior directions |\n| AirfRANS  | Point cloud, inlet velocity, SDF, boundary normals,  interior directions |\n| Blood Flow |  Point cloud, time-varying pressure values at the aortic inlets and outlets | \n| Bounded Navier-Stokes | Regular grid,  obstacle locations, initial physical field|\n| Darcy Flow |  Regular grid,  point-wise diffusion coefficient   | \n| Elasticity  | Point cloud of elasticity material (include void) | \n| Plasticity  | Point cloud of plasticity material, time indexing, shape of rigid die | \n\n### **Q6: The spectral clustering for a new object with significantly new shape will be time-consuming.**\n\nDuring the testing phase, spectral clustering is no longer required. Instead, we perform point-wise matching between the new shape and the reference shape via optimal transport (OT) to directly retrieve the corresponding clustering result.\nAs shown in Table 9 on page 20, the time of point matching via optimal transport is 0.012s for the \nShape-Net Car dataset.\nThe runtime of the OT depends on the number of discrete points on the car surface and is independent of the geometric differences between shapes.\nIt is important to contextualize this overhead relative to traditional numerical solvers. For example, a conventional $k$–$\\varepsilon$ turbulence simulation typically takes around 50 minutes for a single prediction of the Shape-Net Car."}, {"review_id": "O0wxDyp9Ib", "reviewer": "Reviewer_1P4P", "summary": "The paper focuses on improving learning solution operators for PDE on various geometries. First, the authors proposed a tokenization method  based on the eigenvalue of Beltrami-Laplacian operator $\\Delta$ on boundary $\\partial \\Omega$, which characterizes the intrinsic geometry structure of $\\partial \\Omega$. By spectral clustering, the domain is decomposed into subregions. Second, the author proposed an architecture of neural operators that has two components. A coarse attention handles feature of the clustered points from the first step, and a fine attention handles features from original points.\n\nExperiments include three datasets of challenging geometry shapes: Shape-Net Car, AirfRANS, Blood Flow. The proposed method outperforms all baselines.", "questions": "N.A.", "limitations": "N.A.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength**\n\nThe idea of tokenizing points by spectral clustering is novel and reasonable. The paper is nicely presented and easy to follow.\n\n**Weakness**\n\nThe proposed dual-attention structure causes overhead parameters and computation (see Table 9). One alternative would be simply using clustering as additional feature. Also, such clustering feature should naturally be utilized by the \"slicing\" structure of Transolver. Hence, I strongly recommend the authors to conduct an experiment of combining your tokenization technique with Transolver.", "was_revised": true, "final_justification": "I thank the author for the additional experiment in their rebuttal to respond my concerns. \n\nIt seems that the proposed method has certain advantage particularly near boundary and contact surface due to its geometric embedding. It seems that the embedding technique can be complimentary to other neural operators as well, e.g., the performance of Transolver is improved in the additional experiment. \n\nAs for the proposed architecture, a design of local-global splitting is recently getting popular, for example, [1]. The proposed method has achieved a good balance between accuracy and computational efficiency due to local (within patch) attention and coarse (patch-wise) attention.\n\nTherefore, the overall contribution of this paper is positive. I will change my score to 5.\n\n[1] Liu-Schiaffini, M., Berner, J., Bonev, B., Kurth, T., Azizzadenesheli, K., & Anandkumar, A. (2024). Neural operators with localized integral and differential kernels. arXiv preprint arXiv:2402.16845.", "rebuttal": "### **Q1: Using clustering as an additional feature.**\n\nWe thank the reviewer for their thoughtful comments. In response, we designed a dedicated experiment to evaluate this aspect.\nWe first perform aligned boundary clustering on car surfaces following our method (Lines 153–165, page 5). The resulting spectral clustering labels are encoded into one-hot vectors, which are then passed through a two-layer MLP to match the dimensionality of Transolver’s initial features. These features are added to Transolver’s original initial features before entering the Transformer block. All other components of Transolver remain unchanged.\n\n\nTable R4-1.  Comparative results of Transolver variants incorporating our clustering with different clusters on Shape-Net Car.\n\n| Model              | Vol ↓ | Surf ↓ | $C_D$ ↓ | $ \\rho_D $ ↑ |\n|-------------------|-----------|--------|--------------|----------------|\n| Transolver   |  0.0228 |   0.0793  |  0.0129   |  0.9916     |\n| SpiderSolver (Ours)  | 0.0210  | 0.0738 | 0.0100  | 0.9928   |\n| Transolver with 4 clusters   |  0.0212 | 0.0776    |  0.0126 |   0.9894     |\n| Transolver with 6 clusters   |  0.0220 |   0.0789  |  0.0123 |    0.9913   |\n| Transolver with 8 clusters   |  0.0217 |  0.0782   |   0.0119|   0.9906    |\n| Transolver with 10 clusters   | 0.0212  |   0.0778  | 0.0115  |   0.9919    |\n\nAs shown in Table R4-1, integrating our clustering features into the initial features of Transolver yields a slight performance improvement. However, the gain remains limited, and the model still falls short of the performance achieved by our full SpiderSolver. \nThis indicates that while clustering provides useful geometric priors, the holistic design of SpiderSolver\n(including Spiderweb tokenization, fine- and coarse-grained attention) is essential for fully leveraging boundary-aware representations.\n\n### **Q2: Combine our tokenization with Transolver's slicing structure.**\n\nWe appreciate the reviewer’s insightful suggestion and accordingly conducted a targeted experiment.\nWe replace Transolver’s learnable slicing mechanism with our deterministic Spiderweb tokenization to define the patch structure. We set $m_I = 8$ and  $m_B = 4$, resulting in the same number of patches as used in Transolver.\nThe remaining components of Transolver, including the architecture and training procedure, are kept unchanged. \n\nTable R4-2.  Comparative results of Transolver variants incorporating Spiderweb tokenization on Shape-Net Car.\n\n| Model              | Vol ↓ | Surf ↓ | $C_D$ ↓ | $\\rho_D$ ↑ |\n|-------------------|-----------|--------|--------------|----------------|\n| Transolver   |  0.0228 |   0.0793  |  0.0129   |  0.9916     |\n| SpiderSolver (Ours)  | 0.0210  | 0.0738 | 0.0100  | 0.9928   |\n| Transolver with Spiderweb tokenization   |  0.0223 | 0.0752    | 0.0112  |    0.9920   |\n\nAs shown in Table R4-2, \nreplacing Transolver’s original slicing mechanism with our spiderweb tokenization improves its performance. However, it still underperforms compared to our full SpiderSolver model. \nThis performance gap arises because SpiderSolver further incorporates fine-grained attention to model interactions between boundary points and nearby interior points, enabling more precise information flow across the domain.\n\n###  **Conclusion and future work.**\n\nWe thank the reviewer 1P4P for offering a valuable perspective on our model architecture. The suggestions provide promising directions for future exploration, and we plan to investigate these ideas in subsequent work.\nBeyond the reviewer’s suggestions, we have also identified several potential avenues for future research based on the overall findings and limitations of our current study.\n\n\nIn this paper, we propose SpiderSolver, a transformer-based model tailored for solving PDEs over complex domain geometries.\nAt the core of SpiderSolver is the novel spiderweb tokenization, which enables flexible geometric decomposition of the domain. The model integrates both coarse-grained attention (across region-level tokens) and fine-grained attention (between boundary and interior points), capturing multiscale interactions essential to PDE dynamics.\n\nWe originally evaluated SpiderSolver on five representative benchmarks: ShapeNet Car, AirfRANS, Blood Flow, Bounded Navier-Stokes, and Darcy Flow.\nWith the newly added experiments on the Elasticity and Plasticity datasets, our evaluation now spans seven benchmarks with diverse physics and geometries.\nAcross all these settings, SpiderSolver consistently achieves state-of-the-art performance.\nThese results demonstrate the generality and practical value of our approach, highlighting its potential for solving real-world scientific problems involving complex domains. \n\nIn future work, we plan to extend the framework to multiple classes of shape\n boundaries, possibly by making the network parameters aware to different geometric classes. We\n also plan to extend it to be equivariant to the geometry transform of the PDE boundary and domain."}]

Author rebuttal:

Meta-review: The paper addresses the challenging problem of solving PDEs on complex geometries with a geometry-aware transformer architecture. Reviewers uniformly found the work to be novel and timely, praising its strong motivation, solid technical design, and clear connections between geometric priors and the model structure. The empirical results, though limited in scale, demonstrate consistent improvements over competitive baselines and support the claimed advantages. The rebuttal further clarified technical points and addressed questions on scalability and robustness to the reviewers’ satisfaction. Overall, given the positive feedback across the board and the clear contributions in both methodology and application, I recommend acceptance.
Decision: Accept (poster)