Neural P$^3$M: A Long-Range Interaction Modeling Enhancer for Geometric GNNs

Authors: yusong  wang
chaoran  cheng
shaoning  li
yuxuan  ren
bin  shao
ge  liu
pheng-ann  heng
nanning  zheng
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 15:54:07

Abstract: Geometric graph neural networks (GNNs) have emerged as powerful tools for modeling molecular geometry. However, they encounter limitations in effectively capturing long-range interactions in large molecular systems. To address this challenge, we introduce **Neural P$^3$M**, a versatile enhancer of geometric GNNs to expand the scope of their capabilities by incorporating mesh points alongside atoms and reimaging traditional mathematical operations in a trainable manner. Neural P$^3$M exhibits flexibility across a wide range of molecular systems and demonstrates remarkable accuracy in predicting energies and forces, outperforming on benchmarks such as the MD22 dataset. 
It also achieves an average improvement of 22% on the OE62 dataset while integrating with various architectures. Codes are available at https://github.com/OnlyLoveKFC/Neural_P3M.

Main paper: 
 introduction: 
 Prevailing geometric graph neural networks (GNNs) have demonstrated remarkable capabilities in capturing the geometric information inherent within molecular graphs. Not only do they accelerate the computational efficiency compared to traditional Density Functional Theory (DFT) methods for molecules, but also hold the promise of achieving high-level accuracy in predicting crucial molecular properties such as energy and forces [2, 18, 23] . Despite their success in modeling small molecules, limitations still persist in extending these methods to larger molecular structures and systems governed by periodic boundary conditions (PBC). Current methods [16, 1] excel in approximating the shortrange interactions, which encapsulate interactions among local atom groups within a defined distance cutoff, characterized by a rapid decay in real space. The primary obstacle lies in effectively capturing long-range interactions within these complex systems.Several attempts have been undertaken to incorporate long-range physical interactions into geometric GNNs. Early studies [19, 21] combined physical equations, such as Coulomb's law, with models tailored for short-range interactions. Conversely, recent advancements are steering towards the development of sophisticated models capable of learning long-range interactions directly from data. One such strategy is the spatial-based method, exemplified by LSRM [13] . It utilizes specific fragmentation algorithms like BRICS [5] to fragment molecules into discrete groups in real space. The long-range interactions are thereby captured in a hierarchical manner by facilitating message passing between the fragments and atoms. Another strategy is the spectral-based method [12, 24] , which treats the long-range parts in the reciprocal space following the concepts of Ewald summation [4] . The long-range parts exhibit a rapid decay instead in the reciprocal space, which enables efficient evaluation with a frequency cutoff.Following traditional computational chemistry, an intuitive direction would be to mesh up the Ewald summation, harnessing fast Fourier transformation (FFT) for acceleration. While this poses a nontrivial problem, a rich of established works represented by Particle-Particle Particle-Mesh (P 3 M) [11] provide a solid foundation for such undertakings. In this work, inspired by the underlying unified concepts [6] behind these FFT-accelerated methods, we propose a novel perspective by integrating atom and mesh into neural networks. To be concrete, we reimage the traditional mathematical operations in mesh-based methods in a trainable manner, laying the foundation of our new framework, termed Neural P 3 M (Fig. 1 ). Neural P 3 M is designed to be a versatile enhancer, compatible with a wide range of existing models. In contrast to LSRM, Neural P 3 M framework remains unconstrained to any fragmentation algorithm, and hence enhances its flexibility across diverse molecular systems. Different from Ewald MP, Neural P 3 M explicitly incorporates mesh representations, thereby offering discrete resolutions necessary for formulating long-range terms. Additionally, it incorporates the exchange of information between short-range and long-range terms at the atom and mesh scales. Moreover, our proposed framework exhibits theoretical efficiency surpassing that of Ewald MP due to the reduced computational complexity afforded by FFT.Long-rangeNeural P 3 M (Atom-Mesh) (Mesh-Mesh) (Atom-Atom) We evaluate our framework on several benchmarks by integrating a variety of geometric GNNs.Neural P 3 M achieves the state-of-the-art performance on the MD22 dataset [3] and Ag dataset [16] when combined with ViSNet [23] . It consistently demonstrates improvements in energy mean absolute errors (MAEs), achieving an average reduction of 22% on the OE62 dataset [20] . In summary, our contributions can be summarized as follows:• Framework. We propose a novel framework Neural P 3 M to capture short-range and long-range interactions at both atom and mesh scale.• Enhancement and Versatility. Neural P 3 M exhibits compatibility and significant improvements with short-range-centric methods on the Ag, MD22 and OE62 benchmarks.• Flexibility. Neural P 3 M is well-suited for diverse molecular systems without any constraints. 
preliminary: 
 Ewald Summation Ewald summation is a widely used technique in calculations of long-range interactions in periodic systems [7] . Specifically, consider the pair-wise electrostatic potential as ψ(r ij ) = 1/∥r ij ∥ 2 . The total electrostatic potential energy E can be evaluated as the infinite summation over pairs under the periodic boundary condition (PBC) asE = 1 2 n N i=1 N ′ j=1 ρ i (r)ρ j (r ′ )ψ(∥r -r ′ + n • c∥ 2 )d 3 rd 3 r ′ = 1 2 N i=1 ρ i (r) ϕ [i] (r)d 3 r (1)where ρ i (r) is charge density, c is the cell vector, and N is the number of atoms in a cell. The ′ summation is introduced to exclude the term j = i, if and only if n = 0. ϕ [i] (r) represents the potential generated by all particles excluding the particle i. A continuous partition function that delays rapidly with respect to the distance is used to separate the short-range and long-range terms. One standard approach is to partition the contributions based on the error function erf:EQUATIONwhere β is a fixed constant. We assume the charge density is described by the delta function as point charges, i.e. ρ i (r) = q i δ(r -r i ). With the rapid delay of the partition function, it is safe to assume convergence by only considering the interaction pairs within a specific cutoff distance asEQUATIONwhere E is the set of atom pairs within the cutoff distance. By the Parseval's theorem, the corresponding long-range term can be expressed as the summation in the Fourier domain asEQUATION)where V is the volume of the unit cell and g(m) = 4π/∥m∥ 2 2 are the Fourier transformed Green function of the Coulomb potential 1/∥r∥ 2 , and γ(m) = exp(-∥m∥ 2 2 /4β 2 ). The Fourier-transformed charge density ρ(m) is defined asEQUATIONThe frequency vector m can be truncated as the long-range term quickly converges in the Fourier domain. As the long-range term introduces the self-interaction energy, a correction term is also applied to the final potential energy asE self = - 1 2 N i=1 ρ i (r) ϕ lr i (r)d 3 r = - β √ π N i=1 q 2 i (6)Meshing up the Ewald Summation The traditional Ewald summation method has a computational complexity of O(N 2 ), which becomes impractical for large-scale systems. A common approach to accelerate the process is to employ FFT. Currently, a variety of mesh-based implementations are available. While they differ in their implementations, they share a similar conceptual foundation [6] .Initially, point charges (particles) with their continuous coordinates, must be scattered onto grid-based charge densities (meshes). The charge densities on meshes are interpolated using charge assignment function W to ensure a finite support for summation:EQUATIONwhere V grid is the volume of the discrete grid to ensure that ρ M is a density. Once we have discrete grid-based charge densities, we need to modify Eq.4 to accommodate discrete mesh points. According to the proof in Appendix B, Eq.4 can be rewritten as the convolution in the real space:EQUATION)where G is referred to influence function following Hockney and Eastwood [11] and ⋆ is the convolution operation. The discrete approximation for E lr can be expressed in a corresponding manner as follows:EQUATIONwhere, V is the set of mesh points. By altering the standard influence function G to accommodate different charge assignment functions, one can develop distinct algorithms. Subsequently, FFT is employed to accelerate the convolution process. Following the calculation of the energy, forces on particles can be determined by differentiation, either in the real space or Fourier space. Alternatively, forces can also be derived by differentiating on meshes and then applying a back-interpolation technique to assign forces to particles.The adaptation of FFT to the Ewald summation has been quite enlightening. We will delve into a detailed examination of the correlation between our Neural P 3 M and these mesh-based techniques in the subsequent section. 
method: 
 We are interested in learning the energies and forces of 3D molecules, potentially under the assumption of the periodic boundary condition. Specifically, consider a 3D molecule represented as a point cloud G = {x a i , z i } i∈U with atom coordinates x a and atom types z, we want to learn the molecule-level energy Ê(G) and atom-level forces F (G). Different from previous work [12] which utilizes the vanilla Ewald summation in the Fourier domain, our framework is mesh-based which provides discrete structural information and allows for information flow between long-range and short-range representations. Our fundamental concept is akin to these mesh-based methods mentioned in Section 2. We use short-range blocks on atoms to capture bonded terms and non-bonded short-range terms while applying long-range blocks on meshes to handle long-range terms. We enable the transfer of information between atoms and meshes via the representation assignment. A pseudocode for the Neural P 3 M block is provided in Appendix D.1 to enhance understanding. We further elaborate on the Neural P 3 M architecture as follows.Firstly, we construct meshes on which long-range interactions can be captured. In periodic systems such as crystals, the cell is naturally delineated. For non-periodic systems, we adopt the approach used by prevalent quantum chemistry software, which involves padding the bounding box with a specified margin to define the cell. Detailed information about the construction of the cell can be found in Appendix C. The coordinates of mesh points x m i,j,k can be described as:EQUATIONwhere c = [c x , c y , c z ] ⊤ is the cell vector and N x , N y , N z is the number of discretizations along each dimension. For convenience, we can regard meshes as a point cloud with a single subscript for the index as {x m i } i∈V . Once coordinates of mesh points are established, we can proceed to construct a short-range atomic radius graph and a bipartite radius graph between atoms and meshes as follows:EQUATIONEQUATION)where U is the atom set and V is the mesh set. Specifically, for periodic systems, the edges are also obtained by considering possible cross-boundary connections. The atom representation h 0 i is initialized as:EQUATIONThe initial mesh representation, denoted as m 0 i , is obtained by averaging the representations of all neighboring atoms on the atom-mesh bipartite graph:m 0 i = 1 |M(i)| j∈M(i) h 0 j (14)where M(i) represents the set of neighboring nodes connected to mesh node i within E assign . The edge features in both E short and E assign can be expanded via a set of radial basis functions (RBF):EQUATION3.3 Neural P 3 M Block Short-range Block The short-range block (Fig. 2(c )) updates the atomic representations using a graph neural network that is either SE(3)-equivariant or invariant. This process can be generally expressed as follows:EQUATIONWe noted that the usage of radius graphs inherits the localization assumptions in geometric GNNs and any node is only able to aggregation information from its direct geometric neighbors in one short-range block. Therefore, we naturally interpret it as capturing the short-range contribution to the energy and forces. As this part involves only atoms, we call such a module Atom2Atom which corresponds to the particle-particle part (short-range term) in the P 3 M.The long-range block (Fig. 2(d )) updates mesh representations globally. Recalling Eq.9, the key aspect is to devise the influence function G and utilize FFT along with the convolution theorem for efficient computation of the convolution. Within our framework, we parameterize G directly in the Fourier domain, and the updated mesh representations can be described as:EQUATIONwhere F, F -1 are the Fourier transform and inverse Fourier transform on the discretized mesh, respectively. σ is the activation function. W long and G are the learnable weights that parameterize the operator in the real space and Fourier space. If we consider m as a continuous function v(m), our formulation coincides with the Fourier neural operators (FNOs) on the discretized continuous function. Similarly, as the long-range block only involves interactions within meshes, we call it Mesh2Mesh.Representation Assignment The representation assignment block (Fig. 2 (e)) allows for information flow between atom representations and mesh representations, effectively mixing short-range and longrange terms to obtain a more comprehensive descriptor of the molecule. By parameterizing the charge assignment function W in Eq.7 and substituting the charge density with the atom representation hl j , we can derive the continuous filter convolution (CFconv) proposed in SchNet [17] . To elaborate further, we get additional mesh representations as:EQUATIONThis Atom2Mesh module can be regarded as the information flow from the short-range part to the long-range part. Similarly, the Mesh2Atom module takes the same input and geometric graph but outputs additional atom representations (a ← m) l , which could be viewed as the back-interpolation operation. It allows for the information flow in the inverse direction, from the long-range part to the short-range part. The long-range Mesh2Mesh module together with the Atom2Mesh and Mesh2Atom modules corresponds to the particle-mesh part (long-range term) in the P 3 M.Ultimately, as shown in Fig. 2 (b), we merge the information updated by each part itself with the normalized information received from the other part, and we also incorporate a residual connection to obtain the final output as:EQUATIONEQUATIONAs we are interested in the prediction of molecule-level energies and atom-level forces, an additional decoder is applied to the final atom representations h L and mesh representations m L to get the atom-wise energies h out and mesh-wise energies m out . We follow previous work to assume the additive property of energy to sum all atom-wise energies as the short part of the molecule energyEQUATIONWe also sum all mesh-wise energies as the long part of the molecule energy Êlong .EQUATIONThe final potential energy is calculated as: Ê = Êlong + Êshort . Furthermore, although direct prediction of forces is possible, we instead use the negative gradient of the energy as the prediction of forces: F = -∇ x Ê. The final training objective is a weighted loss between energy and force:EQUATION4 ExperimentQuestion: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes] Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: All datasets used in this paper are publicly and freely accessible. We have included sufficient instructions to the datasets and our experimental settings in Section 4. We will open-source for reproducibility once our paper gets published.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setup: 
 In this section, we conduct comprehensive validations of our Neural P 3 M framework using diverse datasets and configurations. First, we intuitively demonstrate the necessity of incorporating longrange interactions through a toy dataset Ag used in Allegro [16] . Subsequently, we integrate various geometric GNNs [17, 9, 18, 8, 23] with our Neural P 3 M framework on two prevalent datasets OE62 [20] and MD22 [3] to demonstrate versatility and effectiveness. All results are evaluated using mean absolute error (MAE) on test sets, and the baseline results are sourced directly from the corresponding papers. Unless stated otherwise, almost all hyperparameters align with the baseline GNNs. For a more comprehensive overview of hyperparameter settings and implementation details, please refer to the Appendix D and F. The Ag dataset comprises 1,159 structures sampled from a 1,111K AIMD simulation [16] . These structures were generated from a bulk facecentered-cubic lattice with a vacancy, encompassing 71 atoms subject to periodic boundary conditions. For consistency with Allegro, we randomly split them into 950 structures for training, 50 structures for validation and the remaining structures for testing.As shown in Fig. 3 , compared to the strictly local Allegro model, ViS-Net, which has only one layer, offers slightly improved force prediction, yet the energy prediction significantly deteriorates. This may be caused by the fact that the model can only perform message passing once, with a lack of long-range interactions. Long-range interactions can be complemented in theory by raising the cutoff from 4.0 Å to 12.0 Å, but this does not work in practice, because it could potentially lead to information over squashing problems, as mentioned in LSRM [13] . When ViSNet with a single layer is integrated into our framework, long-range interactions can be effectively captured, significantly improving the accuracy of energy and force predictions compared to the vanilla ViSNet and Allegro. This toy experiment intuitively demonstrates the critical need to incorporate long-range interactions and emphasizes the significance of a well-crafted methodology in incorporating them.The MD22 dataset [3] consists of MD trajectory datasets, which present challenges due to their larger system sizes, ranging from 42 to 370 atoms. The number of structures in each molecule dataset ranges from 5,032 to 85,109. We calculate the diameter of each molecule, defined as the average of the maximum distance between any two atoms within a molecule. The smallest diameter observed is approximately 10.75 Å, while the largest molecule measures about 32.39 Å. We train a separate model for each molecule and randomly split the dataset according to sGDML [3] .Table 1 demonstrates the results of the ViSNet model incorporating with our Neural P 3 M framework (ViSNet-NP 3 M for short) on MD22. ViSNet-NP 3 M achieves the state-of-the-art (SoTA) performance on both energy and force predictions across the four largest molecules and also achieves the lowest mean absolute error (MAE) for energy or force predictions in the remaining three smaller molecules. When compared to the vanilla ViSNet, ViSNet-NP 3 M showed an average improvement of 34.6% and 21.2% in energy and force prediction, respectively. Notably, our framework exhibits a more substantial improvement when compared to ViSNet-LSRM and ViSNet-Ewald, both of which utilize ViSNet as the short-range model. As shown in Appendix Table 5 , another state-of-the-art model, Equiformer, when integrated with our Neural P 3 M framework, also demonstrates significant enhancements to the short-range model itself. These impressive results highlight our framework's ability to effectively improve the learning of potential long-range interactions in large molecules.It's worth noting that for the two supramolecules that cannot be fragmented by LSRM, our Neural P 3 M achieves a significant performance improvement in energy prediction, with a 57.48% increase for the double-walled nanotube and a 16.07% increase for the buckyball catcher. This suggests that our Neural P 3 M is a general solution for various molecules, which is not limited by traditional fragmentation methods like BRICS.We further take our analysis by incorporating four prevailing geometric GNNs including SchNet [17] , PaiNN [18] , DimeNet++ [9] , and GemNet-T [8] on the OE62 dataset [20] to confirm the framework's versatility. The OE62 dataset consists of about 62,000 large organic molecules, each with the energy calculated by Density Functional Theory (DFT) . The structures within the OE62 dataset are nonperiodic yet can span large spatial dimensions, exceeding 20 Å. The dataset is strictly split into train, validation, and test set according to Ewald MP [12] . The same dataset preprocessing process as Ewald MP is also applied.The numerical results presented in Table 2 and Appendix Table 6 indicates that the Neural P 3 M framework, which combines four models, delivers more performance gains than Ewald MP and LSRM when using the same hyperparameters. Additionally, our framework exhibits a faster computation time than Ewald MP, likely due to the efficiency of FFT implementation by Pytorch. An unexpected observation is the speed performance of DimeNet++. Given that DimeNet++ does not inherently facilitate message passing between atom embeddings, Ewald MP compensates by integrating longrange interactions in each output block. In contrast, our approach exchanges short-range and longrange representations in each layer, which might account for our marginally slower speeds compared to Ewald MP. We also provide detailed profiling results for the number of model parameters, GPU memory usage, and other relevant metrics in Appendix G. For more details on the implementation on the four models, please refer to the Appendix D.We first investigate the impact of the Atom2Mesh and Mesh2Atom modules. We remove the Atom2Mesh module from the original model to avoid the information flow from short-range blocks to long-range blocks and vice versa. Table 3 demonstrates that both modules contribute synergistically to the model's overall performance. The results illustrate the necessity of enabling information exchange between the long-range and short-range blocks.Compared to the vanilla model, our framework introduces only two new hyperparameters: the assignment cutoff distance between mesh points and atoms, denoted as r assign , and the number of mesh points in each dimension, represented as N x , N y , N z . We find that the selection of the number of mesh points is crucial for the model's final performance. As illustrated in Appendix Fig. 4 (b), the mean absolute error (MAE) in energy increases with the number of mesh points, while the forward computation time also extends. This decline in performance may be attributed to instances where each atom is assigned to multiple mesh points simultaneously.As such occurrences become more frequent, the model may struggle to effectively learn the appropriate assignment rules. In practice, we typically set the cutoff distance to either 4.0 or 5.0 Å, ensuring that the product of the number of mesh points and the cutoff distance is approximately equivalent to the cell size in each dimension.Additionally, we provide further ablation studies on the impact of the assignment cutoff distance (without the k-NN graph) to examine the effects of multiple assignments. As shown in Appendix Fig. 4 (c), all experiments exhibit a slight decrease in performance due to multiple assignments. However, an appropriately chosen cutoff (4 or 5 Å) still yields relatively optimal results. Notably, the results do not worsen further as the assignment cutoff increases. We hypothesize that this may be because a larger assignment cutoff creates a broader neighborhood environment, facilitating the learning of the assignment function with a fixed number of meshes, thereby alleviating the challenges associated with multiple assignments. 
related work: 
 Geometric Graph Neural Networks Geometric graph neural networks preserve equivariance toward the rigid transformation in space, which can be categorized according to their emphasis on specific types of structural features and their respective methods of integration. SchNet [17] stands out as the pioneering approach to applying continuous filter convolution on molecular distances. Subsequently, DimeNet++ [9] and GemNet [8] explicitly incorporate angles and dihedrals using Fourier-Bessel functions. To address the computational complexity associated with angles extractions, PaiNN [18] and ViSNet [23] adopt the density trick and reduce the complexity to linear time.Additionally, many works are based on high-order geometric tensors [2, 1, 16, 22] , which ensure rigorous theoretical guarantees of equivariance through the use of Clebsch-Gordan product. Despite these advancements, all these existing methods are constrained to the local atomic environment, and are unable to approximate the long-range interactions. Hence, there is an urgent need for a comprehensive framework to address this challenge.Long-range Interaction Modeling Incorporating long-range interactions into a short-range model is challenging. Early studies attempted to compensate these long-range effects by integrating physical equations with either hand-crafted terms [19] or predicted charges [21] . While, recent works have shifted towards creating carefully designed models that can directly learn long-range interactions from data. The LSRM framework [13] , for instance, captures long-range interactions in real space by using specific algorithms to fragment molecules into discrete groups and models their interactions hierarchically. Other methods [12, 24, 15] handle long-range components in reciprocal space, employing concepts like Ewald summation [4] . Our approach differs from these works by introducing the discretized meshes and facilitating the exchange of information between long-range and short-range components. 
conclusion: 
 In this paper, we introduce a novel framework, termed Neural P 3 M, designed to enhance the longrange interaction modeling for various geometric GNNs. In addition, Neural P 3 M stands out by not being confined to any specific fragmentation approach, making it adaptable to various molecular systems. Neural P 3 M achieves significant performance improvement on prevalent benchmarks by capturing short-range and long-range interactions at both atom and mesh scales, and enabling the exchange of information between them.Limitation and Societal Impacts: The limitation of our study is that it does not thoroughly investigate the impact of the number of meshes, nor does it explore potentially more effective methods for modeling long-range interactions beyond FFT. Nonetheless, our paper offers the community a fresh perspective on molecular geometry modeling. Our proposed Neural P 3 M framework is an extensive of existing geometric GNNs for energy and force prediction of molecules. The prediction of molecular energies and forces has diverse applications in downstream tasks including molecular dynamics simulation and molecular property prediction. As our framework better captures the long-range interaction within the molecule, it can potentially accelerate the pharmaceutical discovery and understanding of diverse molecules that have positive impacts on treating diseases. On the other hand, we are also aware of the potential negative impact if the model is misused, as our understanding of different molecules is still very limited. We will work closely with both the machine learning and the science community to ensure the proper usage of our model for the good of society.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: We have included all details of the model architecture, data processing, and hyperparameter settings in Appendix D and F for reproducing and understanding our results.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No]Justification: We followed previous work to report performance on a single seed. We also fixed the seed for reproducibility instead of averaging across multiple seeds.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: Our paper does not release any high-risk data or models.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We have adequately and properly cited and credited the datasets and models used in this papaer.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: Our paper does not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: Our paper does not involve crowdsourcing or research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA] Justification: Our paper does not involve crowdsourcing or research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. 
 
 Appendix: A Notations The index of blocks r, x aThe coordinates of particles (atoms) r p , x mThe coordinates of the meshes r ijThe displacement vector between the particle i and j cThe cell vectors mThe frequency vectors erfThe error function ρ(•)The charge density of the particle δ(•)The delta function qThe point charges ρ M (•)The charge density of the mesh point ψ(•)The pair-wise electrostatic potential ϕ(•)The potential generated by all particlesϕ [i] (•)The potential generated by all particles excluding the particle i ϕ i (•)The potential generated by the particle i. g, γ, ρThe Fourier transformed function g, γ, ρ ⋆The convolution operation NThe number of particles in a unit cell WThe charge assignment function GThe influence function V grid , VThe volume of the discrete grid and cell.The ground truth and prediction of potential energy F, FThe ground truth and prediction of atomic forces zThe atom types U, VThe set of atoms and meshes N x , N y , N zThe number of discretizations along each dimension x, y, z r short , r assignThe cutoff distance of radius graphs E short , E assignThe edge set of radius graphsN (i), M(i)The neighboring nodes of the target atom (mesh) node. hThe atom representations mThe mesh representations fThe edge representations F, F -1The Fourier transformer and inverse Fourier transform on the discretized mesh (FFT and IFFT) ∥ • ∥ 2The 2-norm of a vector σ(•)The activation function (SiLU)The short-range graph neural network (learnable)W longThe weights in long-range block in real space (learnable) GThe weights in long-range block in Fourier space (learnable) W m←a , W a←mThe weights representation assignment (learnable)The multi layer perception (learnable)The layer normalization λ E , λ FThe weights in the loss between energy and forces B Detailed Derivation of Eq.8Let's start our derivation by replacing the the square of the ρ(m)'s modulus as the product of itself with its conjugate:EQUATIONWe can then confidently interchange the summation symbols and put the normalization factor 1 V within the summations as:EQUATIONUsing convolution theory, which states that the convolution of two functions is the pointwise product of their Fourier transforms, it becomes clear that the expression in parentheses represents the inverse Fourier transform. Consequently, we can rewrite the expression as follows:EQUATIONWe refer g ⋆ γ as the smeared Coulomb Green function G (influence function), and altering it when assigning charges with different charge assignment function W .Cell construction is trivial for periodic systems like crystals, as a canonical cell can always be assigned. We now describe the cell construction for non-periodic systems. Given a set of atom coordinates {x i } n i=1 , we first derive a canonical coordinate frame U as the eigenvectors of the covariance matrix:EQUATION)where Λ is the diagonal matrix of the eigenvalues of the covariance matrix, X ∈ R n×3 is the coordinate matrix, and µ = n i=1 x i /n. For any rotation matrix R and X ′ = XR, it is easy to see that U ′ = RU is a new eigenvector matrix for the new covariance matrix. Therefore, we use the canonical coordinates as X = (X -µ)U ⊤ which is invariant under global translation and rotation. After the transformation, the principle components of the coordinates now align with the coordinate frame. We can define the cell vectors to follow the directions of the coordinate with the cell length defined by the maximum coordinate span with additional padding d on both sides:EQUATIONwhere x, ỹ, z are coordinate components of the transformed molecules. In practice, we used a d = 0.5Å. The mesh coordinates are obtained via Eq.10 and the final atom coordinates are obtained by moving the molecule inside the cell as:EQUATIONThere are rare cases when the molecule exhibits high symmetry. However, as we only consider different atom types and treat the same type of atoms as indistinguishable, the final molecule and mesh are also indistinguishable and unique in this sense.In this section, we first provide the pseudocode for the Neural P 3 M block to facilitate understanding of our framework, followed by a detailed explanation of the implementation. We emphasize the distinct integration strategies required by the varying inputs and outputs of short-range geometric GNNs. For detailed insights into the specific implementations within geometric GNNs, we direct readers to the original paper.The pseudocode for the Neural P 3 M block is presented in Algorithm 1 as a general framework for iteratively and interdependently updating the atom features h and mesh features m. The GNN in the algorithm can incorporate most geometric GNN frameworks that use node features, the atom graph E short , and edge features f short as input to update the node features. The FNO serves as the long-range block, updating mesh features according to Eq.17. The representation assignment then calculates the relevant features based on the assignment graph E assign and edge features f assign . Finally, the overall representation is updated using information from both the short-range and long-range blocks.Algorithm 1 Neural P 3 M block 1: Input: Atom feature h l , mesh feature m l , atom graph E short , assignment graph E assign and edge features f short , f assign .2: hl ← GNN(h l , E short , f short ) ▷ Atom2Atom (Short-range) 3: ml ← FNO(m l ) ▷ Mesh2Mesh (Long-range) 4: (a ← m) l i ← MLP j∈A(i) ml j • W l a←m f assign ij ▷ Mesh2Atom (Repr. Assignment) 5: (m ← a) l i ← MLP j∈M(i) hl j • W l m←a f assign ij ▷ Atom2Mesh (Repr. Assignment) 6: h l+1 ← h l + hl + LN((a ← m) l ) ▷ Mesh2Atom (Update) 7: m l+1 ← m l + ml + LN((m ← a) l ) ▷ Atom2Mesh (Update) 8: return h l+1 , m l+1SchNet [17] utilized continuous graph convolutional kernels generated from edge features of radial basis functions (RBFs) to capture the geometric information of interatomic distances. In each Neural P 3 M Block, the atom representations h l and mesh representations m l are initially subjected to layer normalization before being processed by a SchNet Block and an FNO Block, respectively. hl = SchNet Block(LN(h l ), ...) (33) ml = FNO(LN(m l )) (34) Following this, the representation assignment block updates these representations separately.EQUATION(a ← m) l = Mesh2Atom( ml , ...) (36) The exchanged representations are then normalized and combined with their corresponding updated representations via an addition operation. Finally, we employ residual concatenation to obtain the final representation:EQUATION)EQUATIONPaiNN [18] is an equivariant graph neural network based on scalar-vector interactions. Each hidden state is described by a tuple of scalar representations h l and vector representations vec l and updated as follows: hl , ṽ ec l = PaiNN Block(LN(h l ), vec l ...)We only use scalar representations to exchange information with mesh representations, vector representations can also get long range information when interacting with scalars. The implementations of other parts are consistent with SchNet.DimeNet++ [9] is an improved version of the original DimeNet [10] architecture. In addition to distance, it further leverages the geometric information of any angles formed by three nodes and applies 2D spherical Bessel functions to embed the angles. Thus, the hidden state f l of DimeNet++ is at the edge level. To exchange information between atoms and meshes, we need to aggregate the edge-level representations to the node-level representations as follows:EQUATIONEQUATIONThe subsequent implementations are consistent with SchNet, while in order to obtain the final edgelevel representations, we combine the atom representations on both sides of the edge, and finally update it as follows:EQUATIONEQUATION)EQUATIONGemNet [8] further extends DimeNet to incorporate geometric information of dihedral angles formed by four atoms and applies high-order Bessel functions to embed the dihedral angles. However, since the computational complexity of quadruplets is too high, GemNet-T used in this paper still uses triplets, which can be viewed as more complex DimeNet. GemNet updates both atom-level and edge-level representations as follows:EQUATIONWe use node-level representations to exchange information with mesh representations, and subsequent implementations are consistent with SchNet. The final representations are updated as follows:EQUATIONEQUATIONEQUATIONIt should be noted that updates are made solely at the edge-level representations to prevent information redundancy. Our observations indicate that edge-level representations are predominantly parts of GemNet, hence, we focused our updates there. Additionally, we remove the scaling factor from our implementation.ViSNet [23] is an upgraded version of PaiNN, also utilizing scalar-vector interactions that can describe angles, dihedral angles, and improper angles in linear time complexity. When training ViSNet on the MD22 dataset, we find that ViSNet suffers from unstable training when learning rate is relatively large, so we slightly modified the implementation. Unlike the first 4 models, instead of exchanging information using the representations after updating, we use the input representations directly after layer normalization:EQUATIONEQUATION(a ← m) l = Mesh2Atom(LN(m l ), ...)(52) The final representations are modified as follows:EQUATIONEQUATIONEQUATIONThis modification is similar to altering from post-normalization to pre-normalization in the standard Transformer.We further evaluate the performance of the Equiformer model integrated with our Neural P 3 M on the two largest molecules on MD22, as shown in Table 5 . The results demonstrate our framework's consistent improvement over mainstream state-of-the-art models, further highlighting the compatibility of our approach. Due to the diversity of OE62, the fragmentation algorithm used by LSRM is not suitable for all molecules in this dataset. Nevertheless, for the sake of a thorough comparison, we applied filtering to select a subset of molecules and used this slightly modified dataset to evaluate LSRM's performance on OE62. The results in Table 6 indicate that while LSRM outperforms the baseline, its performance remains below that of our Neural P 3 M. We describe our ablation experiments in detail here. We chose the simplest SchNet model and evaluate on the OE62 dataset.We set the cutoff distance distance between atoms and mesh points r assign , to a constant value of 4.0 Å and fix the number of discretizations to 3. Subsequently, we incrementally removed either Atom2Mesh, Mesh2Atom, or both from the original architecture to prevent information exchange between short-range and long-range blocks, thereby assessing the impact of these modules. The results are presented in Table 3 . We ensure that the cutoff between the atoms and the mesh points r assign is constant (4.0 Å) and that the number of discretizations is the same in all three directions, i.e., N x = N y = N z . The results of the forward time performance and performance with the number of mesh points are shown in Fig. 4 (a) and (b).We conduct ablation studies to examine the impact of the assignment cutoff distance. For the performance results reported in Table 2 , we use a combination of a radius graph and k-NN graph, setting the maximum number of neighbors to 5, which generally minimizes multiple assignments. To assess the effect of multiple assignments, this ablation experiment uses only the radius graph, varying the assignment cutoff distance from 3 to 10. The performance results based on different assignment cutoff distances are shown in Fig. 4(c ).NeurIPS Paper Checklist Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 


Decision: This paper consider the incorporation of mesh points into geometric GNNs for materials science. The reviewers largely agree that this is a valuable contribution that is well-supported, and I believe concerns have been adequately addressed. I accordingly recommend acceptance.

Reviews: review: paper_summary: paper_summary: This work proposes Neural P3M, a framework that enhances geometric GNNs by integrating mesh points and leveraging Fast Fourier Transform (FFT) for efficient computation of long-range interactions. The framework includes short-range and long-range interaction modeling and enables the exchange of information between atom and mesh scales. Neural P3M improves the prediction of energies and forces in large molecular systems, achieving good performance on benchmarks like MD22 and OE62.
strength_weakness: strength_weakness: 1. I feel the overall impact and novelty of this work are limited, given that several works have adopted Ewald summation in geometric GNNs. This work enhanced this concept by introducing FFT for accelerated Ewald summation, which is a common way in traditional simulations and is also already mentioned as a possible direction in [1]. 

2. The experimental part is not comprehensive. The experiment on MD22 doesn't have a comparison to [1], and LSRM is not being compared on OE62. Besides, it is important to also compare the memory consumption between different approaches when dealing with long-range interactions, since in many real-world problems we hope to capture long-range interactions in large molecular systems.

[1] Ewald-based long-range message passing for molecular graphs. ICML 2023
questions: questions: 1. What do the "Embeddings", "Cutoff", and "SchNet-LR" mean in Table 2?

2. What is the computational complexity of the proposed method?
limitations: limitations: The authors have addressed the limitations and potential societal impacts.
paper_summary: paper_summary: The paper introduces Neural P3M, a framework designed to enhance geometric GNNs by incorporating mesh points alongside atoms and transforming traditional mathematical operations into trainable components. The mesh representations offers discrete resolutions necessary for formulating long-range terms. The Neural P3M is also efficient due to the reduced computational complexity afforded by FFT. 
The paper starts with highlighting the importance of long-range terms, which are absent and inefficient in previous works. The paper then explain the preliminary of Ewald summation and the meshing up methods with detailed formulas. The novel methods and neural network blocks are presented in the next section and evaluated with various models and datasets.
The experiment results show significant improvements. When integrated with ViSNet, Neural P3M achieves state-of-the-art performance in energy and force predictions across several large molecules, outperforming other leading models. The framework, combined with models like SchNet, PaiNN, DimeNet++, and GemNet-T, demonstrates enhanced performance and faster computation times compared to related works.
Neural P3M provides a robust framework for enhancing geometric GNNs, enabling them to capture long-range interactions efficiently. The framework's adaptability to various molecular systems and its demonstrated performance improvements on key benchmarks make it a significant contribution to the field. The study also highlights areas for future research, such as optimizing the number of mesh points and exploring alternatives to FFT for modeling long-range interactions.
strength_weakness: strength_weakness: 1. The framework's reliance on complex mathematical operations and integration of mesh-based methods with GNNs can make implementation challenging. Researchers and practitioners may require significant expertise in both GNNs and numerical methods to effectively utilize Neural P3M. It is better to remove some unnecessary equations in Section 2 and 3, or move them to the appendix or references.
2. Additionally, you can present the P3M Blocks by some pseudocode. 
3. The framework's need to handle both atomic and mesh representations simultaneously may lead to increased memory usage, which could be a bottleneck for handling large datasets or systems with limited hardware capabilities. You can present the GPU memory usage while training model with or without Neural P3M.
4. There are some trivial mistakes in Table 2. If the higher Rel. is better, the up arrows should be used. The best runtime should also be highlighted.
questions: questions: 1. Does Neural P3M have some restrictions of geometric GNN models? Or it can be combined with most GNNs?
2. What are the throughputs of Short-Range Block and Long-Range Block? Is the Long-Range Block slower than Shaor-Range Block? Which block is the main bottleneck or is it possible to improve the performance?
3. The variants of the same model usually share some common layers or blocks. Is it possible to reuse or frozen some common layers in a pre-trainded model, and fine-tune the newly added blocks, such as Long-Range Block? So the training performance will be further improved.
limitations: limitations: 1. Distributed training is important and efficient while training the general models with large datasets. However, the distributed training of Neural P3M is not evaluated, and the proposed models and results are now limited to one GPU.
2. Profiler results and the number of parameters in each variant are not presented.
paper_summary: paper_summary: This work introduces a long-range focused GNN that utilizes the combination of atom and mesh representations. The mesh framework in this work is trainable and unconstrained to the fragmentation algorithm. Results demonstrate superior performances across MD22, Ag, and OE62 datasets.
strength_weakness: strength_weakness: - Few of the results seem carefully designed to demonstrate improvements through this approach which makes the overall impact unclear. I've mentioned more concrete points related to this as questions and I can be convinced otherwise post rebuttal/discussion.
- The anonymous link to the code is not in the paper.
questions: questions: - What’s the reason for choosing VisNet to add Neural P3M in Table 1 instead of Equiformer/MACE which has better baselines?
- MD22 results seem to have very carefully designed hyperparameters (cutoffs and number of meshes in each direction) in Table 4 of Appendix E.2. Do you have thoughts on the feasibility of finding these hyperparams on datasets with larger diversity? Additionally, what do the results look like if you just have a uniform hyperparameter across all molecules?
- Is it possible to add results with LSRM in Table 2 results as well? Was there a reason for not including that?
- In Table 2, how does the GPU memory usage compare across these methods? Just out of curiosity, I wanted to know what were the max. batch sizes you were able to fit across these methods. That would most likely make throughput comparison with Ewald even more impressive (assuming a smaller batch size fit for Neural P3M)
- How important is it to have both atom2mesh and mesh2atom modules? Do you get similar performances by just having one and maybe some speedup? Some of the prior literature in charge density models only have mesh2atoms message passing.
limitations: limitations: Yes, this work does mention its limitations and potential negative impact.
paper_summary: paper_summary: The accurate modeling of both short-range and long-range interactions in molecular systems is crucial for predicting properties like molecular energies and forces with high precision. However, traditional Geometric Graph Neural Networks (GNNs) fail to capture such interaction. The paper introduces Neural $\text{P}^3$M, a framework that enhances the capabilities of GNNs via meshing up the Ewald summation. Compared to previous Ewald-based GNNs, the Neural$\text{P}^3$M further enhances the model by incorporating mesh and atom representation in a trainable manner. The paper conducts extensive empirical studies and justifies the effectiveness of the proposed method.
strength_weakness: strength_weakness: - The major contribution of the Neural $\text{P}^3$ M seems to lie in the representation assignment. However, the paper does provide enough ablation studies on the effectiveness of this component. It is unclear whether changing the Fourier Transform to FFT is the main reason for the efficiency.
- The introduction of different concepts can be further improved. For example, the detailed introduction of the Ewald summation in Section 2 mostly is not explicitly ref in the following section. However, Equation 17, which is crucial for the Neural $\text{P}^3$ M, is not elaborated properly, especially for the influence function G and the connection to Equation 9.
- Despite the parameterized assigning function, the representation assignment still assigns each atom to multiple mesh points when the number of mesh points increases. This would post drawbacks on the performance and forward time. The author could consider discussing the different choices of the distance cut-off, as this hyperparameter could differ across datasets.
questions: questions: 1. Discuss the different choices for the distance cut-off.
2. Could the author explain more about the connection between Equation 17 and Equation 9 and the concept of the parameterizing strategy of influence function G?
3. Provide ablation studies on the proposed component and the discussion on the effectiveness of the FFT.
limitations: limitations: It is interesting to involve mesh points to help capture the long-range interaction. The author could extend the idea to other molecules like protein or DNA.

Comments: title: Conclusion for Author-Reviewer Discussion Period
Gratitude for Your Review
Reply to Rebuttal
We are looking forward to your reply
We are looking forward to your reply
Gratitude for Your Review
Gratitude for Your Review and Increasing the score
Response
comment: Dear Reviewers,

We sincerely thank you for your insightful and constructive feedback throughout the review process. Your thoughtful comments have helped make our work more comprehensive and concrete. We are grateful for your recognition of our efforts, contributions, and novelty in our Neural P$^3$M framework, and we are glad that our rebuttals with additional 28 experiments and profiling results have successfully addressed your questions and comments.

We especially enjoyed the interactive discussions during the rebuttal period and we always look forward to the possibility of engaging in a productive dialogue with the reviewers. We are grateful to three reviewers for their active engagement in the discussion period and for their unanimous recognition of our contributions. 

We will make sure to add the additional experiment results and ablation studies in our revised manuscript. Again, we thank you for your valuable input and for helping us bring this research to fruition.

Warm Regards,

The Authors
Thank you once again for your recognition of our work and for the time and effort you've dedicated to the review process. We are very happy to receive your feedback and are pleased to know that we've addressed your concerns. Your insightful questions and suggestions are intriguing and will certainly inspire our future work.
Thank you for your answers to my questions, suggestions, and weaknesses. Your answers have clarified my doubts to a considerable extent. I will not increase the rate of the paper.
Dear Reviewer 7zWV,

We are thankful for your valuable feedback and the recognition you have given our manuscript.

**As the deadline for discussion nears**, we would like to gently remind you of the thorough response we have crafted to address the issues you highlighted. We have carefully addressed **all the points you raised in the Weaknesses**, and have conducted **additional experiments** to answer your interesting questions. We find your questions heuristic and are happy to further engagement on these topics.

If you believe we have resolved the issues and adequately answered your questions, we would be grateful if you could reconsider your score. We welcome any further questions or discussions you may wish to have.

Warm regards,

The Authors
Dear Reviewer M1Fm,

Thank you for your insightful feedback on our manuscript.

**As the deadline for discussion nears**, we wish to remind you that we have provided **a comprehensive response** to address the concerns you raised. With respect to the novelty of our work, we would like to highlight that the other reviewers have recognized our contributions and novelty. It seems there may have been a **misunderstanding** regarding this during the initial review, and we highlighted our novelty in the previous rebuttal. Furthermore, in response to your comments on our experiments, we have made the necessary enhancements.

If you believe we have resolved the issues, we would be grateful if you could reconsider your score. We welcome any further questions or discussions you may wish to have.

Warm regards,

The Authors
We really appreciate your recognition of our work and your decision. Thank you for the effort and time you put into your review!
Thank you for taking the time. We really appreciate your decision. Your support is invaluable in improving our work.
Thanks for your detailed response. It addresses all my problems!
I thank the authors for their rebuttal. Overall, all my raised concerns have been addressed. As a result, I've increased my score.