Paper: A Structured Matrix Method for Nonequispaced Neural Operators (ICLR-2024)

Reviewer comment (Novelty):
The primary weakness of this manuscript is in its framed contribution and presentation around the "structured matrix method," with a secondary issue being the baselines used and some details of the numerical experiments.

Author response:
We start by addressing the reviewer's concerns about **framed contribution of the article and presentation around the "structured matrix method"**: In this regard, we would like to point out a possible misunderstanding about our contributions. While we agree completely with the reviewer that there has been considerable body of work on *non-uniform FFTs* and related algorithms in the signal processing literature, among others, our aim in the paper was NOT to propose a new method for evaluating the discrete Fourier transform efficiently on non-uniform grids. Rather, our goal was to propose a computationally efficient realization of widely popular neural operator architectures, such as FNO, which are currently restricted to uniform grids, on arbitrary point clouds. To this end, we needed to replace the FFT module inside the Fourier layers of FNO with an alternative that can be computationally efficient on unstructured grids. We do so by leveraging the following: 
    i) the observation, backed by both empirical work as in the original FNO paper as well as more recently by theory (see ArXiv:2210.01074 and ArXiv:2304.13221), that FNO type models only require a fixed small (compared to the number of points) number of Fourier modes for accurate operator approximation 
    ii) the DFT structure, being a special Vandermonde matrix, makes it relatively straightforward to obtain a fast algorithm by directly evaluating exponential sums 
    iii) availability of highly optimized functions within PyTorch such as *torch.bmm* and *torch.matmul* for tensor operations that allows us to computationally realize the fast DFT on arbitrary point clouds. 
    We combine all these ingredients into the SMM algorithm and demonstrate through considerable empirical evaluation that it provides a simple, accurate and fast realization of neural operators on arbitrary point clouds. This is the main contribution of this paper and it has been acknowledged as such by other reviewers. We have modified the main text to more explicitly frame our contributions and to mention that we do not propose a new non-uniform FFT algorithm but rather employ a suitable one within the framework of operator learning. This should also be contrasted with existing algorithms that extend neural operators to arbitrary grids. These approaches rely on either interpolation to regular grids or learnable geometric transformations between point clouds and regular grids, making them difficult to train and generalize and/or inaccurate. We also demonstrate through our experiments that our approach leads to a significant improvement over the state of the art methods in this regard (see Table 1).

Reviewer comment (Novelty):
For the so-called Type 1 NUFFTs considered, it is well known that direct evaluation of the exponential sums is more efficient if only a small number of modes are needed; this also applies to the equispaced transform (with a different crossover point). Standard NUFFT libraries explicitly state this (e.g., the "Do I even need a NUFFT?" section in finufft documentation). Therefore, it is unclear what the contribution is here. Additionally, the presentation of the method in Section 2 is not particularly clear, especially when moving to general point clouds where the notation seems to reduce to just 2 sampling points, and there are cleaner ways to write the transform.

Author response:
Furthermore, we agree that there are nicer ways to write such transforms, such as through summation notation, which is often done in literature. However, as the implementation of this transform within the context of machine learning through a PyTorch library is of special interest, we choose to describe the matrices themselves, as opposed to the transform. For problems where points vary between training samples, it is also paramount that these matrices may be efficiently constructed, mainly by avoiding **for loops**. Therefore, we hope to provide a general illustration of the structure of the matrix itself, as well as the implementation available in the provided code so that others planning to implement FNO-based methods on nonequispaced point-clouds will be able to tailor such an approach to their specific application.

Reviewer comment (Theory):
The section titled "Inverse Transformations" is misleading. The manuscript appears to conflate the adjoint transform with an inverse transform, which are different concepts. This incorrect usage occurs throughout the text. Similarly, in the related work section, the discussion about the lack of a "direct" inverse NUFFT is misleading. Type 1 and Type 2 NUFFTs are not inverses of each other (unlike in the equispaced case), and the "inverse" in contexts like imaging problems is typically understood in a least squares sense, which is not something the authors provide.

Author response:
With regards to the reviewers' comments on **terminology and transform representations**, we start by acknowledging that some terms were indeed used loosely and imprecisely in our original manuscript. In particular, the adjoint and the inverse are indeed not equivalent in the nonuniform case. This was an abuse of terminology on our part. For uniform discretizations, the inverse DFT is used for the Fourier neural operator in the discrete case, and we chose terminology to reflect this practice. To clarify the underlying process, we compute the product of an integral kernel matrix and the data in the frequency space, where we then use the conjugate transpose of the Vandermonde-structured matrix to transform our data back from the frequency domain to the spatial domain. The Vandermonde-structured matrices have an extremely ill-conditioned inverse, which is why explicit computation of the inverse of Vandermonde-structured matrices is rarely done. Thus, to be more clear to the reader, we have now changed the term "inverse transformation" to the "backward transformation" to denote the mapping back from frequency to physical space.

Reviewer comment (Theory):
The discussion of computational complexity is lacking. Complexity should be expressed in terms of the number of modes (m) and the number of sample points (n), e.g., $\mathcal{O}(mn)$. A claim could then be made that m is often constant as n grows, leading to linear complexity—but this is questionable, since in more complex problems both m and n would likely increase. Additionally, given the focus on the number of modes, it is surprising that the experiments do not specify how many modes were used in each case, which would be easy to include and informative.

Author response:
In response to the reviewer's comments on **computational complexity and model specifications**, we agree it is more precise to include the number of modes in the computational complexity, and we have made this update in the revised version (See **SM** Section A.3). We have also included the number of modes for each benchmark in **SM** Table 4. As observed from the table, the number of Fourier modes is always in the range $12-32$ and is significantly less than the number of collocation points.

The reviewer also raises an interesting question regarding the empirical relations between the number of modes and the number of points. A priori, it is not possible to say how many modes must be selected for a problem of given complexity or resolution - these must be determined through model selection, varying the number of modes, and choosing the model that minimizes the error over the validation set. This model selection process generally reveals that there is a point where increasing the number of modes results in worse performance for a given data set. With regard to the features that may arise in complex sets of PDEs or high-resolution data, the skip connections and nonlinear activation functions of the model architecture are much more capable of resolving such features. The Fourier layers serve to propagate information throughout the domain. These heuristic considerations have been backed by recent theoretical and empirical work in ArXiv:2210.01074 and ArXiv:2304.13221 and references therein.

Reviewer comment (Experiments):
In the experiments, it is unclear why an established NUFFT library like finufft is not used as a comparison point instead of cubic interpolation. Furthermore, the "full grid" comparisons seem misleading; it appears that the "Full Grid" baseline uses more modes rather than the reduced number. If so, the baseline should be using the reduced number and, if that number is small enough, the sums could be directly evaluated.

Author response:
We would like to address the reviewer's question on the **Full Grid baselines**. These "Full Grid" baselines use an integral kernel with the same number of modes as the SMM comparisons. Nonetheless, the FFT used in this baseline computes all the modes. The number of modes is indeed low enough for the sum to be directly evaluated, specifically through a PyTorch matrix multiplication function, and this is one of our primary contributions in this paper. This simple approach, as well as its extension to nonequispaced point clouds, is the key point of our paper and leads to a concrete realization of neural operators on arbitrary point clouds, a feature that was long promised in the literature but, to the best of our knowledge, is only provided in our paper.

Reviewer comment (Novelty):
Some relevant prior work is missing from the references. In particular:  
- A parallel non-uniform fast Fourier transform library based on an “exponential of semicircle” kernel. A. H. Barnett, J. F. Magland, and L. af Klinteberg. SIAM J. Sci. Comput. 41(5), C479-C504 (2019).  
- Dutt, Alok, and Vladimir Rokhlin. "Fast Fourier transforms for nonequispaced data." SIAM Journal on Scientific computing 14.6 (1993): 1368-1393.  
- Beylkin, Gregory. "On the fast Fourier transform of functions with singularities." Applied and Computational Harmonic Analysis 2.4 (1995): 363-381.

Author response:
We thank the reviewer for pointing out relevant references, and we have added them to the list of references in the revised version of our paper.

Reviewer comment (Novelty):
The paper’s framing remains misleading about its true contribution. The main result is a comprehensive experimental illustration that for certain FNO problems, a small enough number of non-equispaced modes makes direct evaluation efficient. However, the manuscript’s title and framing suggest an algorithmic contribution in spectral transforms, which does not exist beyond implementation. Given how small m is, direct evaluation is the obvious approach.

Author response:
At the outset, we would like to sincerely thank the reviewer for their constructive feedback as well as for raising their rating for our paper. The reviewer is rightfully concerned about the **framing of our contributions** and we take this opportunity to further clarify our rationale for the choices that we had made in the original version of our paper. As we have stated in the paper and as the reviewer (and other reviewers) have acknowledged that our primary contribution is indeed the *realization of FNO (and related neural operators) on arbitrary point distributions*, a topic of great importance in the learning of partial differential equations. Clearly, our paper should not be construed as a contribution in the general area of *structured transforms* and we do apologize if our framing of the title, abstract and related work section led to a perception of contributions to structured transforms. We would like to rectify this misperception as much as possible in the **CRV**, if accepted. To this end, following the discussion with all the reviewers, we propose to change the title of the **CRV**, if accepted, to **Fourier Neural Operators on arbitrary point clouds** or **Neural Operators on arbitrary grids**. This will directly frame as contribution in terms of operator learning, rather than to structured transforms. Consequently, we will modify the abstract as well as the introduction, related work and discussion to very clearly state our exact contribution as we have framed it in our rebuttal to your evaluation. We will pay special attention to your comments on the related work section in this regard. 

 It is also pertinent here to point out that our choice of the title was motivated by the need to differentiate our paper from papers such as "Nonequispaced Fourier PDE Solvers" and "Geometry-Aware FNOs", which claim to realize FNO on arbitrary point clouds but do so via either interpolation or learnable transformations between irregular and regular grids. This is NOT what we do, so we chose a title that reflects this difference but we believe that the proposed modified title will reflect our true contributions in a much sharper manner. 

Finally, the reviewer rightly points out the use of FFT and Spherical harmonics as *red herrings*. In the original versions of FNO, the idea to use FFT was to reduce computational complexity as the authors thought that a large number of modes are necessary. This thinking has been persistent in the community ever since although many people have observed empirically that in practice, very low number of modes actually performs on par or better than large number of Fourier modes and the expressivity in FNO comes from large lifting layers, residual connections, and nonlinear activation functions. Theoretical understanding of this phenomena is much more recent, see ArXiv:2210.01074 and ArXiv:2304.13221. But it is precisely these observations that have motivated our approach which based on direct sum evaluation and leads to a very simple method for extending FNO to arbitrary point clouds, which is both more accurate as well as more computationally efficient than state of the art baselines. One could argue that this observation of ours was quite straightforward but to the best of our knowledge, we were first to do so and also to implement it at scale. This constitutes the core of our contributions in this paper and we aim to state it even more clearly and explicitly in a **CRV**, if accepted. 

We thank the reviewer again for your very constructive feedback that has helped us to better frame our contributions and significantly improve the quality of the paper. We hope to have addressed your concerns satisfactorily through this reply and if so, we kindly request you to upgrade your evaluation accordingly.

Reviewer comment (Novelty):
The manuscript could be more appropriately framed as “a realization of FNOs on non-uniform grids," with only a brief note on the fact that transforms can be evaluated efficiently due to the small number of modes. As written, it still suggests a contribution in “structured matrix methods” that is not substantiated. NUFFT comparisons are somewhat superfluous, and the long “related work” section is of limited relevance given the true contribution.

Author response:
At the outset, we would like to sincerely thank the reviewer for their constructive feedback as well as for raising their rating for our paper. The reviewer is rightfully concerned about the **framing of our contributions** and we take this opportunity to further clarify our rationale for the choices that we had made in the original version of our paper. As we have stated in the paper and as the reviewer (and other reviewers) have acknowledged that our primary contribution is indeed the *realization of FNO (and related neural operators) on arbitrary point distributions*, a topic of great importance in the learning of partial differential equations. Clearly, our paper should not be construed as a contribution in the general area of *structured transforms* and we do apologize if our framing of the title, abstract and related work section led to a perception of contributions to structured transforms. We would like to rectify this misperception as much as possible in the **CRV**, if accepted. To this end, following the discussion with all the reviewers, we propose to change the title of the **CRV**, if accepted, to **Fourier Neural Operators on arbitrary point clouds** or **Neural Operators on arbitrary grids**. This will directly frame as contribution in terms of operator learning, rather than to structured transforms. Consequently, we will modify the abstract as well as the introduction, related work and discussion to very clearly state our exact contribution as we have framed it in our rebuttal to your evaluation. We will pay special attention to your comments on the related work section in this regard. 

 It is also pertinent here to point out that our choice of the title was motivated by the need to differentiate our paper from papers such as "Nonequispaced Fourier PDE Solvers" and "Geometry-Aware FNOs", which claim to realize FNO on arbitrary point clouds but do so via either interpolation or learnable transformations between irregular and regular grids. This is NOT what we do, so we chose a title that reflects this difference but we believe that the proposed modified title will reflect our true contributions in a much sharper manner. 

Finally, the reviewer rightly points out the use of FFT and Spherical harmonics as *red herrings*. In the original versions of FNO, the idea to use FFT was to reduce computational complexity as the authors thought that a large number of modes are necessary. This thinking has been persistent in the community ever since although many people have observed empirically that in practice, very low number of modes actually performs on par or better than large number of Fourier modes and the expressivity in FNO comes from large lifting layers, residual connections, and nonlinear activation functions. Theoretical understanding of this phenomena is much more recent, see ArXiv:2210.01074 and ArXiv:2304.13221. But it is precisely these observations that have motivated our approach which based on direct sum evaluation and leads to a very simple method for extending FNO to arbitrary point clouds, which is both more accurate as well as more computationally efficient than state of the art baselines. One could argue that this observation of ours was quite straightforward but to the best of our knowledge, we were first to do so and also to implement it at scale. This constitutes the core of our contributions in this paper and we aim to state it even more clearly and explicitly in a **CRV**, if accepted. 

We thank the reviewer again for your very constructive feedback that has helped us to better frame our contributions and significantly improve the quality of the paper. We hope to have addressed your concerns satisfactorily through this reply and if so, we kindly request you to upgrade your evaluation accordingly.

Reviewer comment (Novelty):
From the perspective of structured transforms, it is unclear why there was ever a perceived gap in efficiently using arbitrary grids, since direct evaluation was always an option and likely the best choice given the small m. The pursuit of NUFFTs or FFTs here seems to have been a red-herring.

Author response:
At the outset, we would like to sincerely thank the reviewer for their constructive feedback as well as for raising their rating for our paper. The reviewer is rightfully concerned about the **framing of our contributions** and we take this opportunity to further clarify our rationale for the choices that we had made in the original version of our paper. As we have stated in the paper and as the reviewer (and other reviewers) have acknowledged that our primary contribution is indeed the *realization of FNO (and related neural operators) on arbitrary point distributions*, a topic of great importance in the learning of partial differential equations. Clearly, our paper should not be construed as a contribution in the general area of *structured transforms* and we do apologize if our framing of the title, abstract and related work section led to a perception of contributions to structured transforms. We would like to rectify this misperception as much as possible in the **CRV**, if accepted. To this end, following the discussion with all the reviewers, we propose to change the title of the **CRV**, if accepted, to **Fourier Neural Operators on arbitrary point clouds** or **Neural Operators on arbitrary grids**. This will directly frame as contribution in terms of operator learning, rather than to structured transforms. Consequently, we will modify the abstract as well as the introduction, related work and discussion to very clearly state our exact contribution as we have framed it in our rebuttal to your evaluation. We will pay special attention to your comments on the related work section in this regard. 

 It is also pertinent here to point out that our choice of the title was motivated by the need to differentiate our paper from papers such as "Nonequispaced Fourier PDE Solvers" and "Geometry-Aware FNOs", which claim to realize FNO on arbitrary point clouds but do so via either interpolation or learnable transformations between irregular and regular grids. This is NOT what we do, so we chose a title that reflects this difference but we believe that the proposed modified title will reflect our true contributions in a much sharper manner. 

Finally, the reviewer rightly points out the use of FFT and Spherical harmonics as *red herrings*. In the original versions of FNO, the idea to use FFT was to reduce computational complexity as the authors thought that a large number of modes are necessary. This thinking has been persistent in the community ever since although many people have observed empirically that in practice, very low number of modes actually performs on par or better than large number of Fourier modes and the expressivity in FNO comes from large lifting layers, residual connections, and nonlinear activation functions. Theoretical understanding of this phenomena is much more recent, see ArXiv:2210.01074 and ArXiv:2304.13221. But it is precisely these observations that have motivated our approach which based on direct sum evaluation and leads to a very simple method for extending FNO to arbitrary point clouds, which is both more accurate as well as more computationally efficient than state of the art baselines. One could argue that this observation of ours was quite straightforward but to the best of our knowledge, we were first to do so and also to implement it at scale. This constitutes the core of our contributions in this paper and we aim to state it even more clearly and explicitly in a **CRV**, if accepted. 

We thank the reviewer again for your very constructive feedback that has helped us to better frame our contributions and significantly improve the quality of the paper. We hope to have addressed your concerns satisfactorily through this reply and if so, we kindly request you to upgrade your evaluation accordingly.

Reviewer comment (Novelty):
The reviewer maintains that the manuscript is still presented as though the main contribution lies in the transforms, which they do not see beyond the simple implementation. They leave it to the area chair to decide if the current version’s framing aligns with the actual contribution.

Author response:
At the outset, we would like to sincerely thank the reviewer for their constructive feedback as well as for raising their rating for our paper. The reviewer is rightfully concerned about the **framing of our contributions** and we take this opportunity to further clarify our rationale for the choices that we had made in the original version of our paper. As we have stated in the paper and as the reviewer (and other reviewers) have acknowledged that our primary contribution is indeed the *realization of FNO (and related neural operators) on arbitrary point distributions*, a topic of great importance in the learning of partial differential equations. Clearly, our paper should not be construed as a contribution in the general area of *structured transforms* and we do apologize if our framing of the title, abstract and related work section led to a perception of contributions to structured transforms. We would like to rectify this misperception as much as possible in the **CRV**, if accepted. To this end, following the discussion with all the reviewers, we propose to change the title of the **CRV**, if accepted, to **Fourier Neural Operators on arbitrary point clouds** or **Neural Operators on arbitrary grids**. This will directly frame as contribution in terms of operator learning, rather than to structured transforms. Consequently, we will modify the abstract as well as the introduction, related work and discussion to very clearly state our exact contribution as we have framed it in our rebuttal to your evaluation. We will pay special attention to your comments on the related work section in this regard. 

 It is also pertinent here to point out that our choice of the title was motivated by the need to differentiate our paper from papers such as "Nonequispaced Fourier PDE Solvers" and "Geometry-Aware FNOs", which claim to realize FNO on arbitrary point clouds but do so via either interpolation or learnable transformations between irregular and regular grids. This is NOT what we do, so we chose a title that reflects this difference but we believe that the proposed modified title will reflect our true contributions in a much sharper manner. 

Finally, the reviewer rightly points out the use of FFT and Spherical harmonics as *red herrings*. In the original versions of FNO, the idea to use FFT was to reduce computational complexity as the authors thought that a large number of modes are necessary. This thinking has been persistent in the community ever since although many people have observed empirically that in practice, very low number of modes actually performs on par or better than large number of Fourier modes and the expressivity in FNO comes from large lifting layers, residual connections, and nonlinear activation functions. Theoretical understanding of this phenomena is much more recent, see ArXiv:2210.01074 and ArXiv:2304.13221. But it is precisely these observations that have motivated our approach which based on direct sum evaluation and leads to a very simple method for extending FNO to arbitrary point clouds, which is both more accurate as well as more computationally efficient than state of the art baselines. One could argue that this observation of ours was quite straightforward but to the best of our knowledge, we were first to do so and also to implement it at scale. This constitutes the core of our contributions in this paper and we aim to state it even more clearly and explicitly in a **CRV**, if accepted. 

We thank the reviewer again for your very constructive feedback that has helped us to better frame our contributions and significantly improve the quality of the paper. We hope to have addressed your concerns satisfactorily through this reply and if so, we kindly request you to upgrade your evaluation accordingly.

Reviewer comment (Presentation):
The authors could start their method discussion by introducing Fourier neural operator first, its mathematical formulation, and then pointing out where the structured matrix method is fitted in.

Author response:
1. The reviewer's point about **introducing more information on FNO** is very well taken. Following your excellent suggestion, we have now added an entire section **SM** A.1, where we explicitly define the FNO. We have also referenced this section in the main text. However, given the space constraints, we could not incorporate a full section on FNO in the main text itself.

Reviewer comment (Presentation):
At present, Figure-1 shows their workflow, however it's hard to visualize the change the proposed method is bringing about in the whole neural operator learning workflow.

Author response:
2. In response to the reviewer's point about **it being hard to visualize the proposed method is bringing about in the whole neural operator learning workflow in Figure 1**, we would like to say that we are primarily providing a more efficient method (SMM) that replaces the FFT and inverse FFT of the Fourier layer of the FNO and allows FNO to be realized on arbitrary point distributions. Other than that, the workflow is unchanged, and the FNO diagram would be identical. We have now explicitly mentioned this fact in the main text of the revised version of our paper.

Reviewer comment (Evaluation):
The text in the results section is a bit misleading and makes it sound as if the proposed method outperforms FNO and SFNO on the regular grid. If I understand correctly, this is the method on the irregular grid, which requires an interpolation step.

Author response:
Regarding the reviewer's questions and comments about **how the proposed method compares to FNO/SFNO on regular grids and putting our method in the right perspective in this regard**: we would start by pointing out we had already provided such a comparison for the one-dimensional Burgers' Eqn (the very first two rows of Table 1) where we had presented the results for FNO on a uniform (equispaced) grid with the underlying Fourier transforms being computed by FFT and compared with the Fourier transforms being computed with SMM. From this table, we see that both methods are completely comparable in terms of training time as well as accuracy. Following your suggestion, we have performed an additional experiment for the 2-D shear layer with data sampled on a (dense) uniform grid for FNO, with FFT and SMM realizing the Fourier transform, respectively. We find that FNO with FFT has a per-epoch training time of $251$ secs with an $L^1$-test error of $6.76\%$. On the other hand, FNO with SMM has a per-epoch training time of $211$ sec and an $L^1$-test error of $6.6\%$. This clearly reinforces the conclusions of the 1-D experiment by demonstrating that, on a regular grid, SMM is completely comparable to FFT as they are simply different computational realizations of the Fourier transform inside FNO. We have now modified the main text of the paper to clearly point out this fact.

Finally for the Shallow water equations on the sphere, we followed the reviewer's suggestion and trained SFNO (as well as FNO) on the underlying regular grid on a sphere. However, we evaluated both methods on randomly distributed points on the Sphere (outputs of SFNO and FNO can be evaluated on any point in the domain), and the resulting test errors are now shown in **SM Table 5**, as well as directly below. Comparing with the results obtained with SFNO-SMM (and FNO-SMM), we see that the errors are very similar whereas the SMM algorithm is faster to train in this case. With these additional results, we hope to have addressed your concerns about putting the results in the correct perspective by comparing with FNO/SFNO on regular grids.

| Method | Training Time (per epoch) | Error over Collocation Points |
|--------|---------------------------|-------------------------------|
| SFNO   | 30s                       | 3.60%                         |
| FNO    | 32s                       | 5.74%                         |

Reviewer comment (Experiments):
It would have been better if the provided metrics were put into context by showing the approximation results of the vanilla FNO/SFNO methods on their respective grids. This would provide a good baseline both for timing and performance results.

Author response:
Regarding the reviewer's questions and comments about **how the proposed method compares to FNO/SFNO on regular grids and putting our method in the right perspective in this regard**: we would start by pointing out we had already provided such a comparison for the one-dimensional Burgers' Eqn (the very first two rows of Table 1) where we had presented the results for FNO on a uniform (equispaced) grid with the underlying Fourier transforms being computed by FFT and compared with the Fourier transforms being computed with SMM. From this table, we see that both methods are completely comparable in terms of training time as well as accuracy. Following your suggestion, we have performed an additional experiment for the 2-D shear layer with data sampled on a (dense) uniform grid for FNO, with FFT and SMM realizing the Fourier transform, respectively. We find that FNO with FFT has a per-epoch training time of $251$ secs with an $L^1$-test error of $6.76\%$. On the other hand, FNO with SMM has a per-epoch training time of $211$ sec and an $L^1$-test error of $6.6\%$. This clearly reinforces the conclusions of the 1-D experiment by demonstrating that, on a regular grid, SMM is completely comparable to FFT as they are simply different computational realizations of the Fourier transform inside FNO. We have now modified the main text of the paper to clearly point out this fact.

Finally for the Shallow water equations on the sphere, we followed the reviewer's suggestion and trained SFNO (as well as FNO) on the underlying regular grid on a sphere. However, we evaluated both methods on randomly distributed points on the Sphere (outputs of SFNO and FNO can be evaluated on any point in the domain), and the resulting test errors are now shown in **SM Table 5**, as well as directly below. Comparing with the results obtained with SFNO-SMM (and FNO-SMM), we see that the errors are very similar whereas the SMM algorithm is faster to train in this case. With these additional results, we hope to have addressed your concerns about putting the results in the correct perspective by comparing with FNO/SFNO on regular grids.

| Method | Training Time (per epoch) | Error over Collocation Points |
|--------|---------------------------|-------------------------------|
| SFNO   | 30s                       | 3.60%                         |
| FNO    | 32s                       | 5.74%                         |

Reviewer comment (Experiments):
Classical methods typically can not perform arbitrarily well on unstructured grids. I expect the same to be true here as the projection will have quadrature errors and you won't be able to evaluate the integrals to high accuracy on arbitrary grids. Have you performed any analysis on how sensitive results are to the choice of collocation points? It would be good to mention this in the paper.

Author response:
Regarding the reviewer's comments and questions about **how sensitive is the proposed method to the choice of collocation points**: the reviewer is correct in pointing out that, in analogy with classical numerical methods, one can expect larger errors on unstructured grids, in general. This is indeed borne out by the experimental evidence already presented in Table 1 for the 1-D Burgers' Eqn example. In the table, we present results on a regular equispaced grid and on an irregular expanding-contracting point distribution (See **SM** Figure 4) to observe that the test error with FNO-SMM increases by almost a factor of $4$ when transiting from a regular to an unstructured grid. We have added a further comment on this issue in the main text and thank the reviewer again for pointing it out to us.

Reviewer comment (Evaluation):
Related to the above question, it would be great to have both timings and L2/L1 errors of the FNO/SFNO on their respective grids to quantify the error of the method and put it into perspective. I expect these results to be better, but I would still welcome this to show the grid-dependence and avoid wrong conclusions for cases where the data is readily available on structured grids.

Author response:
Regarding the reviewer's questions and comments about **how the proposed method compares to FNO/SFNO on regular grids and putting our method in the right perspective in this regard**: we would start by pointing out we had already provided such a comparison for the one-dimensional Burgers' Eqn (the very first two rows of Table 1) where we had presented the results for FNO on a uniform (equispaced) grid with the underlying Fourier transforms being computed by FFT and compared with the Fourier transforms being computed with SMM. From this table, we see that both methods are completely comparable in terms of training time as well as accuracy. Following your suggestion, we have performed an additional experiment for the 2-D shear layer with data sampled on a (dense) uniform grid for FNO, with FFT and SMM realizing the Fourier transform, respectively. We find that FNO with FFT has a per-epoch training time of $251$ secs with an $L^1$-test error of $6.76\%$. On the other hand, FNO with SMM has a per-epoch training time of $211$ sec and an $L^1$-test error of $6.6\%$. This clearly reinforces the conclusions of the 1-D experiment by demonstrating that, on a regular grid, SMM is completely comparable to FFT as they are simply different computational realizations of the Fourier transform inside FNO. We have now modified the main text of the paper to clearly point out this fact.

Finally for the Shallow water equations on the sphere, we followed the reviewer's suggestion and trained SFNO (as well as FNO) on the underlying regular grid on a sphere. However, we evaluated both methods on randomly distributed points on the Sphere (outputs of SFNO and FNO can be evaluated on any point in the domain), and the resulting test errors are now shown in **SM Table 5**, as well as directly below. Comparing with the results obtained with SFNO-SMM (and FNO-SMM), we see that the errors are very similar whereas the SMM algorithm is faster to train in this case. With these additional results, we hope to have addressed your concerns about putting the results in the correct perspective by comparing with FNO/SFNO on regular grids.

| Method | Training Time (per epoch) | Error over Collocation Points |
|--------|---------------------------|-------------------------------|
| SFNO   | 30s                       | 3.60%                         |
| FNO    | 32s                       | 5.74%                         |

Reviewer comment (Novelty):
In both of the original FNO/SFNO papers, it is mentioned that the generalization to arbitrary grids is straightforward by formulating the DFT/SHT on the respective domain. I wish that the authors had stressed this aspect more in the theoretical introduction, as the presented method is essentially a realization of the FNO/SFNO, just on an unstructured grid. This is one of the significant advantages of FNOs, and it is great to see this done in practice.

Author response:
At the outset, we would like to thank the reviewer for pointing out that **our method is essentially the realization of FNO/SFNO on arbitrary grids, something that the literature has promised but never delivered so far.** We fully agree with your assessment in this regard. It is also true that we could have mentioned this point much more explicitly in the main text of our paper. One reason why we did not do so was to emphasize the *even broader scope* of our method. It is not limited just to FNO/SFNO, but in principle, any neural operator that uses spectral transformations (Fourier/Spherical Harmonics/Wavelet, etc.) can be efficiently realized on arbitrary point clouds using our structured matrix method. Already in the paper, we have demonstrated this with FNO, SFNO, UFNO, and FFNO (See Table 1), and the extension to other neural operators of this type is straightforward. However, we agree with the reviewer that FNO (and SFNO) are the most important neural operators of this class in practice. Hence, and following the excellent suggestion of the reviewer, we have explicitly (and repeatedly) pointed out in the revised version that our method realizes FNO/SFNO on arbitrary point clouds. We hope that this explicit message resonates much more with the end users of neural operators in scientific machine learning.

Reviewer comment (Experiments):
In line with what I wrote above, one of the advantages of Neural Operators is that they can be trained on one grid and evaluated on another. Have you experimented with such settings? I would be especially curious in the presented FNO/SFNO case how such an operator would perform compared to one trained on the unstructured grid.

Author response:
In response to the reviewer's point about **advantages of FNO/SFNO is that they can be trained on one grid and evaluated on another**, we would like to say that this is indeed a very important feature of neural operators. As SMM is simply a realization of FNO/SFNO on arbitrary point clouds, the resulting method has exactly the same flexibility. Following the reviewer's excellent suggestion, we have now added a new Table (**SM** Table 5) to demonstrate this feature of the SMM in the specific case of Shallow water equations on a Sphere. To this end, we trained SFNO-SMM on a random point distribution of 5000 points and evaluated the trained model on random point distributions ranging from 4000 to 50000 points. The test errors with a test set of 64 samples are now shown in **SM** Table 5, and we see that the errors remain roughly the same even when the number of collocation points was increased by an order of magnitude. Moreover, we would like to emphasize that both the point distribution as well as the test samples were not seen during training. We believe that this example showcases the ability of the SMM algorithm to generalize to different arbitrary point clouds.

| Approximate No. Points | Error  |
|------------------------|--------|
| 4000                   | 6.36%  |
| 8000                   | 5.64%  |
| 15000                  | 5.44%  |
| 29000                  | 5.39%  |
| 50000                  | 5.43%  |

Furthermore, we would like to emphasize that in the Elasticity and Flow past Airfoil examples (Table 1), the point clouds changed for each sample during training (and testing) (See **SM** Figure 8). Thus, the SMM algorithm has the ability to deal with varying arbitrary point clouds.

We sincerely hope that we have addressed all the reviewer's concerns, particularly about comparing to FNO/SFNO on regular grids and showcasing the ability of SMM to generalize to unseen point distributions, to your satisfaction. We kindly request the reviewer to update your assessment of our paper accordingly.