Radial Basis Operator Networks

## Abstract

Abstract Operator networks are designed to approximate nonlinear operators, which provide mappings between infinite-dimensional spaces such as function spaces. These networks are playing an increasingly important role in machine learning, with their most notable contributions in the field of scientific computing. Their significance stems from their ability to handle the type of data often encountered in scientific applications. For instance, in climate modeling or fluid dynamics, input data typically consists of discretized continuous fields (like temperature distributions or velocity fields). We introduce the radial basis operator network (RBON), which represents a significant advancement as the first operator network capable of learning an operator in both the time domain and frequency domain when adjusted to accept complex-valued inputs. Despite the small, single hidden-layer structure, the RBON boasts small L 2 superscript 𝐿 2 L^{2} italic_L start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT relative test error for both in- and out-of-distribution data (OOD) of less than 1 × 10 − 7 1 superscript 10 7 1\times 10^{-7} 1 × 10 start_POSTSUPERSCRIPT - 7 end_POSTSUPERSCRIPT in some benchmark cases. Moreover, the RBON maintains small error on OOD data from entirely different function classes from the training data.

## 1 Introduction

### 1.1 Background

Traditional feedforward neural networks (FNNs) and radial basis function (RBF) networks have been shown to be universal approximators of functions [, ], meaning they are capable of representing the mapping between *finite* dimensional spaces. Thus, these networks are limited in their design to predicting a measurement acting on a subspace of $\mathbb{R}^{d}$ for some $d\in\mathbb{Z}^{+}$. Operator networks, however, are designed to learn the mapping between *infinite* dimensional spaces; they receive functions as input and produce the corresponding output function. Scientific computing has benefited from using operator networks to enhance or replace numerical computation for the purpose of simulation and forecasting on a wide array of applications to include computational fluid dynamics and weather forecasting [].

The two primary neural operators that demonstrated immediate success are the deep operator network (DeepONet) [] based on the universal approximation theorem in [], and the Fourier neural operator (FNO) []. The basic DeepONet approximates the operator by applying a weighted sum to the product of each of the transformed outputs from two FNN sub-networks. The upper sub-network, or *branch net*, is applied to the input functions while the lower *trunk net* is applied to the querying locations of the output function.

In contrast, the FNO is a particular type of Neural Operator network [], which accepts only input functions (not querying locations for the output) and applies a global transformation on the function input via a more intricate architecture. Motivated by fundamental solutions to partial differential equations (PDEs), the FNO network sums the output of an integral kernel transformation to the input function with the output of a linear transformation. The sum is then passed through a non-linear activation function. To accelerate the integral kernel transformation, the FNO applies a Fourier transform (FT) to the input data, with the FT of the integral kernel assumed as trainable parameters.

Following their initial introduction, several extensions and modifications to FNO and DeepONet were introduced to improve performance in specific contexts. Examples include the Fourier-enhanced DeepONet [] to improve DeepONet’s robustness against Gaussian noise, U-FNO [] and MIONet [] introduce U-Net paths into the Fourier layer of the FNO architecture to improve accuracy for multi-phase flow applications, and model-parallel FNOs [] parallelise the structure of FNO to reduce computation load for high-dimensional data. However, many of these situational improvements did not result in clear error reductions across a variety of contexts, at least not enough to justify the additional complexity in architecture contained in some of the proposed methods. This has changed with the recent introduction of a new neural operator.

The Laplace Neural Operator (LNO) [] has recently become the new standard for operator networks in benchmarking performance. LNO was motivated as an extension to FNO in order to address primary limitations of the FNO, particularly its approximations for transient responses and non-periodic signals. To address this issue, LNO applies the pole-residue method to represent both the transient and steady-state response in the Laplace domain. This translates to better test performance on out-of-distribution (OOD) data in almost all contexts with few exceptions to include a reduced training cost and simpler network architecture. For this reason we have selected LNO as the primary comparison for our new operator network, including a problem scenario from [] with small OOD error for the LNO predictions.

### 1.2 Our Contributions

We propose the radial basis operator network (RBON) based on the universal approximation theorem in []; a novel operator network that is, to the best of our knowledge, the first to be entirely represented with radial basis functions.

- •
The universal approximation result in [] is extended to normalised RBONSs (NRBONs).
- •
The RBON is the first network to successfully learn an operator entirely in both the time domain and frequency domain, by altering the algorithm to accept complex data types.
- •
Despite the simple single-hidden-layer structure, the particular implementation of the RBON within demonstrates impressively small error on both in-distribution (ID) and OOD data, outperforming LNO by several orders of magnitude.
- •
The RBON demonstrates successful results on the first OOD example where the OOD input is an entirely different base function. Typically, OOD input functions for introducing new operator networks are a scaling, shifting, or simple transformation of the input functions used in training.

While operator networks are usually tested only using data generated from known systems, such as in systems of partial differential equations (PDEs), we include a scientific application where the data is real physical measurements and the underlying operator is unknown. This demonstrates the ability of RBON to make accurate forecasts for time-dependent systems, for the purposes of scientific experimentation. The rest of the paper is organised as follows, the theoretical foundation and details regarding the particular implementation are presented in Section 2, which precedes the results of the numerical experimentation first on generated data followed by the observed data in 3, with the discussion and conclusion at the end.

## 2 Methodology

The RBON is a numerical representation, $G_{\dagger}$, for an operator, $G:\mathcal{U}\to\mathcal{V}$, where $\mathcal{U}$ and $\mathcal{V}$ are infinite dimensional spaces, using radial basis functions. Following the work as shown in [], we present, without proof, the universal approximation theorem for such a representation as well as extending the theorem to include NRBONs. The subsequent section details the precise implementation used for the experimental results.

### 2.1 Theoretical Foundation

In distribution theory the Schwartz space, $\mathcal{S}(\mathbb{R}^{d}),$ is the space of rapidly decaying functions that are infinitely differentiable and whose derivatives decay faster than a polynomial. Essentially, these are smooth functions that vanish quickly away from their center. The space containing all linear functionals that act on the Schwartz space is referred to as the space of tempered distributions and is represented symbolically as $\mathcal{S}^{\prime}(\mathbb{R}^{d})$; the prime notation connotes the duality relationship between the spaces. These spaces are for defining the necessary regularity for the radial basis functions used in the approximation.

Noting that $C(A)$ represents all continuous functions defined on $A$, consider the functions $g$ such that,

$$ $g\in C(\mathbb{R})\cap S^{\prime}(\mathbb{R}).$ (1) $$

meaning $g$ is in the space of tempered distributions and is continuous on $\mathbb{R}$. Choosing $\displaystyle||{\bm{x}}||_{\mathbb{R}^{d}}$ to represent the Euclidean norm for $x\in R^{d}$, we can represent a radial basis function acting on ${\bm{x}}$ as

$$ $g(\lambda||{\bm{x}}-\mu||_{\mathbb{R}^{d}})$ $$

for constants $\lambda\in\mathbb{R},\mu\in\mathbb{R}^{d}.$ Then we have the following (see [] for the proof with details).

###### Theorem 2.1

Suppose $g$ is not an even polynomial and satisfies (1), $X$ is a Banach space where $K_{1}\subseteq X,K_{2}\subseteq\mathbb{R}^{d}$ are two compact sets in $X$ and $\mathbb{R}^{d}$ respectively. Suppose also that $\mathcal{U}$ is a compact set in $C(K_{1}),$ $G$ is a nonlinear continuous operator, mapping $\mathcal{U}$ into $C(K_{2})$, then for any small positive $\epsilon$, there are positive integers $M,N,m,$ constants $\xi_{i}^{k},\omega_{k},\lambda_{i}\in\mathbb{R},$ $k\in\{1,\dots,N\},i\in\{1,\dots,M\}$, $m$ points ${\bm{x}}_{1},\dots,{\bm{x}}_{m}\in K_{1}$, ${\bm{c}}_{1},\dots,{\bm{c}}_{N}\in\mathbb{R}^{d},$ such that

$$ $\left|G(u)({\bm{y}})-G^{\dagger}(u^{m})({\bm{y}})\right|<\epsilon$ $$

for every $u\in\mathcal{U}$ and ${\bm{y}}\in K_{2}$, where $u^{m}=(u({\bm{x}}_{1}),\dots,u({\bm{x}}_{m}))$, and

$$ $G^{\dagger}(u^{m})({\bm{y}})=\sum_{i=1}^{M}\sum_{k=1}^{N}\xi_{i}^{k}g(\lambda_ {i}||u^{m}-\mu_{ik}^{m}||_{\mathbb{R}^{m}})g(\omega_{k}||{\bm{y}}-{\bm{c}}_{k} ||_{\mathbb{R}^{d}})$ (2) $$

for $\mu_{ik}^{m}=(\mu_{1k}^{m},\dots,\mu_{mk}^{m}),k=1,\dots,N.$

For $\epsilon$ and $\xi_{i}^{k}$ given as in Theorem 2.1 set

$$ $\tilde{\xi}_{i}^{k}=\xi_{i}^{k}\sum_{i=1}^{M}\sum_{k=1}^{N}g(\lambda_{i}||u^{m }-\mu_{ik}^{m}||_{\mathbb{R}^{m}})g(\omega_{k}||{\bm{y}}-{\bm{c}}_{k}||_{ \mathbb{R}^{d}}),$ (3) $$

and the corollary extending the theorem for the normalised representation follows immediately.

###### Corollary 2.1.1

Under the same assumptions in Theorem $\ref{thm1},$ and with $\xi_{i}^{k}$ as defined in (3), we have

$$ $\left|G(u)({\bm{y}})-\tilde{G}^{\dagger}(u^{m})({\bm{y}})\right|<\epsilon$ $$

where

$$ $\tilde{G}^{\dagger}(u^{m})({\bm{y}})=\sum_{i=1}^{M}\sum_{k=1}^{N}\tilde{\xi}_{ i}^{k}\frac{g(\lambda_{i}||u^{m}-\mu_{ik}^{m}||_{\mathbb{R}^{m}})g(\omega_{k}| |{\bm{y}}-{\bm{c}}_{k}||_{\mathbb{R}^{d}})}{\sum_{i=1}^{M}\sum_{k=1}^{N}g( \lambda_{i}||u^{m}-\mu_{ik}^{m}||_{\mathbb{R}^{m}})g(\omega_{k}||{\bm{y}}-{\bm {c}}_{k}||_{\mathbb{R}^{d}})}.$ (4) $$

The RBON, as represented in (2), comprises two single-layer sub-networks of radial basis functions. This architecture extends the concept of RBF networks to operators, analogous to how DeepONet extended FNNs. The sub-network that processes the function input $u^{m}$ is called the branch net. Here, $u^{m}$ represents the input function $u$ sampled at $m$ point locations, as defined in the theorem. The trunk net, on the other hand, receives inputs corresponding to the domain locations where the network will produce output function values. The diagram in Figure 1 shows the basic RBON architecture with the additional linear output transformation, $\mathcal{L}.$

Figure: Figure 1: The upper network is the branch net that receives the function input. The lower trunk net receives the sensor location input, and $\mathcal{L}$ denotes a linear transformation.
Refer to caption: x1.png

###### Theorem 2.1

###### Corollary 2.1.1

### 2.2 Practical Implementation

Having established the theoretical foundation, we now turn to the practical implementation of our approach. This section outlines the step-by-step process for the realised implementation of both RBON and NRBON. The implementation consists of several key steps that translate our theoretical model into a functional algorithm.

Initially, function inputs $u({\bm{x}})$ are sampled at $m$ point locations as defined in Section 2.1, while sensor location inputs ${\bm{y}}$ are processed directly. For both the trunk and branch networks, we select Gaussian functions for the RBF transformations on the input data. To determine the centers and spreads of the RBFs, we employ K-means clustering [, ] on the input data for each sub-network. The centers obtained from this clustering serve as the RBF centers, and the spread is calculated based on inter-cluster distances.

For weight parameter calculation, we compute the product of the RBF activations for the nodes in both the trunk and branch net. The weights, $\xi_{i}^{k}$, are then determined using the method of least squares. This is done for the branch output of every input $u^{m}_{j}$ for $j$ in the training index, iterated across the trunk output for each query location, ${\bm{y}}$. This process produces different values for the weight parameters for each ${\bm{y}}$ and the final weights are obtained by averaging across all query locations. A linear transform is applied to the final output, the weights of which are solved for directly.

Using K-means to determine the parameter locations for the RBFs limits the number of RBFs in the representation by the size of the training data set. It is worth noting that manually assigning the centers for the RBFs produces satisfactory results, but tends to result in larger error than using K-means. Hence manually assigning centers is only advisable when working with small training sets. Moreover, the majority of the variation in train/test error is mostly due to the varying results from the location parameters determined by the K-means clustering.

Concluding the description of the practical implementation, we note that solving for the network weights tends to yield more accurate predictions when using the Moore-Penrose inverse [, ] instead of a factorisation scheme. Using a least-mean-squares iterative approach also works, but results in weights that on average produce larger error in their predictions.

### 2.3 Learning in the frequency domain

The RBON is designed to learn the operator in the frequency domain as well as in the time domain. The frequency domain is a representation of signals or functions in terms of their frequency components, rather than time. It allows analysis of how signals vary with frequency, providing insight into characteristics like energy, power, and periodicity. The frequency domain is often used to examine cyclic behavior, separate overlapping signals, and simplify certain mathematical operations on signals. Considering that functions have a global representation in the frequency domain, this can have benefits in reducing the variability on the RBONs predictions for OOD data.

Thus, the RBON can be trained on functions in the time domain to approximate the operator $G$ shown in figure 2, or the Fourier transform, $\mathcal{F}$, can be used to convert functions to the frequency domain, in which case the RBON is learning the approximation for what is labeled as $\widehat{G}$ in figure 2. This is especially beneficial for applications where the data is stored in the frequency domain representation.

Figure: Figure 2: The RBON is capable of learning either $G$ or $\widehat{G}$ where $\mathcal{F}$ denotes the Fourier transform and $\zeta,\omega$ are the frequency input.
Refer to caption: x2.png

While algorithmically the RBON follows the same process whether training in the time domain or frequency domain, we denote the application of the RBON to the frequency domain as F-RBON. Numerically, the coded functions for F-RBON accept complex-valued input since functions defined on the frequency domain are stored numerically using complex valued arrays. This allows the F-RBON to capture both magnitude and phase information inherent in frequency domain representations.

## 3 Numerical Experimentation and Results

This section is partitioned into *numerical computing* experiments and a scientific application based solely on data collected from observed measurements. This demonstrates the ability for the RBON to learn the mapping in a variety of contexts including when the mathematical representation for the operator is unknown. We define the numerical computing setting here as scenarios where the data is completely generated from numerical approximations of solutions to mathematical equations. Thus, the operator is known precisely and the results of the RBON can be compared to the numerical approximation of the operator output.

Alternatively, the governing equations for scientific applications are not always known and data is often aggregated from physical measurements. Distinguishing between settings using *generated* data as opposed to *observed* data shows the flexibility of the RBON and its ability to support scientific experimentation and forecasting.

For all the numerical computing experiments, we limited the size of the trunk and branch networks to be no larger than 15 nodes each, capping the number of multiplier parameters in the hidden layer at 225. These restrictions demonstrate the network’s ability to maintain small errors even under incredibly strict size constraints. All code for the RBON learning representation was implemented using the Julia programming language [], chosen for its high-performance numerical computing capabilities, and has been made available at [https://anonymous.4open.science/r/RBON-C1D0/](https://anonymous.4open.science/r/RBON-C1D0/)

### 3.1 Numerical Computing

In each of the following settings, there is a governing system of PDEs defined on a spatio-temporal domain, $\Omega\equiv(0,T)\times(0,L)$ for some final time $T>0$ and length $L>0$. The operator network, $G^{\dagger}$, was trained to learn an operator $G$ within the PDE framework that maps functions representing the initial state or forcing term to the solution over the entire domain.

The input functions for the network for the in distribution data will thus be a family of functions representing an initial state (or forcing term) and parameterized across a specified range of values. ID data was segmented to produce a validation and test set. The validation set was used to optimized the size of the network over a few selected options. The test set provides the in-distribution test error with the out-of-distribution errors based on a set of input functions that have been more significantly altered from the in distribution data.

#### 3.1.1 Wave Equation

Consider the wave equation of the form

$$ $\frac{\partial^{2}u}{\partial t^{2}}=c^{2}\frac{\partial^{2}u}{\partial x^{2}} \,\,\,\,\text{ for }(t,x)\in\Omega,$ $$

where $c$ is the speed of propagation of the wave and $u(t,x)$ models the displacement of a string with Dirichlet boundary conditions. The operator network, $G^{\dagger}$, was trained to learn the mapping, $G$, from the initial state to the solution, $G\colon u_{0}(x)\to u(x,t)$. For the ID data, we particularize the initial condition as

$$ $\displaystyle u_{0}(x)=$ $\displaystyle 2e^{-\left(x-\frac{L}{2}\right)^{2}}+\frac{ax}{L}$ $$

where $a$ is parameterized across the range $[1,4]$ with step size 0.001. The OOD test set uses the same base function for $u_{0}(x)$, but for values of $a$ in the range $[5,5.5]$.

#### 3.1.2 Burgers Equation

Consider the well-known Burgers’ equation

$$ $\frac{\partial u}{\partial t}+u\frac{\partial u}{\partial x}=\nu\frac{\partial ^{2}u}{\partial x^{2}},\,\,\,\text{ for }(t,x)\in\Omega,$ $$

subject to homogeneous Dirichlet boundary conditions and under the following initial conditions $u_{0}(x)=a\sin\pi x$ where $a$ ranges across the interval $[0.1,5]$. The operator learned is thus $G\colon u_{0}(x)\to u(x,t)$ and the RBON is tested on the set of polynomial functions $u_{0}(x)=bx(x-1)$ where $b$ is in the range $[3.5,4.5]$ for the OOD data. Successful testing on the polynomial input after only being trained on the sine function is quite remarkable. The numerical data was generated using the exact solutions as derived in [].

#### 3.1.3 Euler-Bernoulli Beam Equation

The Euler-Lagrange equation for a dynamic Euler-Bernoulli homogeneous beam is

$$ $EI\frac{\partial^{4}u}{\partial x^{4}}+\rho A\frac{\partial^{2}u}{\partial t^{ 2}}=f(t,x)\text{ for }(t,x)\in\Omega,$ $$

where $E$ is Young’s modulus, $I$ is the second moment of the area of the beam’s cross section. The beam’s density is denoted by $\rho$ and the cross-sectional area as $A$. The operator network learns the mapping from the source term to the solution: $G\colon f(t,x)\to u(t,x)$. For the ID data we particularize the source term as $f(t,x)=ae^{-0.05x}(1-10^{2})\sin(10t)$ for $a$ in $[0.05,10]$. The source term for the out of distribution data is $f(t,x)=ae^{-x}(1-10^{2})\sin(10t)$ for $a$ in $[1.24,10.19]$. This scenario was use for testing in []. We include it here for benchmarking purposes, but note we decrease the size of the training set to include in-distribution test error.

**Table 1: Average relative $L^{2}$ error on test data reported with margin of error in parentheses.**
| Network | In/Out | Wave | Burgers | Beam |
| --- | --- | --- | --- | --- |
| RBON | In | $9.4{\text{E}}{-4}(4.9{\text{E}}{-5})$ | $3.6{\text{E}}{-3}(6.0{\text{E}}{-4})$ | ${\bf 4.1E{-8}(3.3E{-6})}$ |
|  | Out | $1.0{\text{E}}{-1}(2.0{\text{E}}{-3})$ | $2.6{\text{E}}{-1}(1.3{\text{E}}{-2})$ | $\bf 1.5E{-8}(2.5E{-7})$ |
| NRBON | In | $1.2{\text{E}}{-5}(9.4{\text{E}}{-7})$ | ${\bf 3.3E{-3}(9.0E{-4})}$ | $1.6\text{E}{-7}(2.4{\text{E}}{-7})$ |
|  | Out | $3.2{\text{E}}{-1}(1.1{\text{E}}{-2})$ | $1.0{\text{E}}{-1}(5.7{\text{E}}{-3})$ | $2.0\text{E}{-8}(4.9{\text{E}}{-9})$ |
| F-RBON | In | ${\bf 3.0{E}{-6}(2.2{E}{-7})}$ | $5.9{\text{E}}{-3}(1.1{\text{E}}{-3})$ | $1.1\text{E}{-1}(1.3\text{E}{-1})$ |
|  | Out | ${\bf 8.6E{-3}(1.7E{-4})}$ | ${\bf 2.3E{-2}(5.5E{-3})}$ | $6.6\text{E}{-2}(7.0\text{E}{-3})$ |
| LNO | In | $5.6{\text{E}}{-1}(1.1{\text{E}}{-3})$ | $1.7{\text{E}}{-1}(4.3{\text{E}}{-4})$ | $1.0\text{E}{-2}(3.9\text{E}{-3})$ |
|  | Out | $5.9{\text{E}}{-1}(9.2{\text{E}}{-4})$ | $2.0{\text{E}}{-1}(8.0{\text{E}}{-6})$ | $6.8\text{E}{-3}(1.5\text{E}{-3})$ |

#### 3.1.4 Results

The results from these experiments can be seen in Table 1, which displays the average $L^{2}$ error for each function in the in-distribution and out-of-distribution test sets. The $L^{2}$ relative error is computed as follows

$$ $L^{2}\,\,rel.\,\,error=\frac{||v^{true}-v^{pred}||_{2}}{||v^{true}||_{2}},$ $$

where $v^{true}$ is the numerical representation of the ground truth at all query points in the space-time domain, and $v^{pred}$ is the network prediction for the same locations.

The margin of error (MOE) displayed in parentheses was calculated by multiplying the point estimate’s standard error with a critical value associated at the 95% confidence level. However, the strict interpretation of the confidence interval is lost due to the lack of independence between observations. Nevertheless, the MOE provides a notion of the variability present in the average error. Each of the different RBON types outperforms the LNO on the majority of the problems, with the NRBON outperforming LNO in every scenario for both ID and OOD data.

### 3.2 Scientific Application

Modeling the relationship between atmospheric $\text{CO}_{2}$ and global temperature is a complex process involving a large number of variables with many of them potentially unknown []. Focusing specifically on an operator that does not have a well-defined mathematical representation, we demonstrate the capacity of the RBON to learn the mapping between monthly atmospheric $\text{CO}_{2}$ measurements and both local and average global monthly temperatures. This provides a template for prediction and forecasting with the RBON based on collected data.

For this section, the RBON is used to learn the operators

$$ $\displaystyle G_{avg}\colon$ $\displaystyle u(t)\to T^{avg}(t),$ $\displaystyle G_{loc}\colon$ $\displaystyle u(t)\to T^{loc}(t)$ $$

where $u$ represents the atmospheric $\text{CO}_{2}$ defined for $t$ in a given time interval, and $T^{avg}$ represents the average global temperature as published in []. The function $T^{loc}$ is local temperature readings at the same site location where the $\text{CO}_{2}$ data was collected. Specifically, atmospheric $\text{CO}_{2}$ concentrations (ppm) derived from in situ air measurements at the well known Mauna Loa, Observatory, Hawaii []. The local temperature readings are much more variable than the global average and hence less easily predicted.

The nature of the operators $G_{avg}$ and $G_{loc}$ is expected to evolve in time due to fluctuations in other contributing factors, however, when continuously updating the RBON with new data, the predictions become quite accurate. While, it is possible to feed the $\text{CO}_{2}$ readings into the network as one function, the centers for the RBFs must be set manually as the K-means algorithm requires at least two function inputs. Instead, it is preferable to parameterize the functions across the years such that $t\in\{1,2,\dots,12\}$ with each number corresponding to the month of the year the measurement was taken. Then operators are thus more accurately represented as

$$ $\displaystyle G_{avg}\colon$ $\displaystyle u_{n}(t)\to T_{n}^{avg}(t),$ $\displaystyle G_{loc}\colon$ $\displaystyle u_{n}(t)\to T_{n}^{loc}(t)$ $$

where $n$ corresponds to a specific year. Training on the historical data, omitting years with incomplete data, yields remarkable accuracy in the RBON predictions as shown in Table 2.

#### 3.2.1 Results

The results in Table 2 demonstrate the ability of a RBON to accurately predict both local monthly average temperatures and global average temperatures. We trained both RBON and NRBON networks on available historical data, omitting the most recent two or five years from the training set to test the models’ forecasting accuracy.

Based solely on monthly $\text{CO}_{2}$ measurements and the month encoding for querying the output temperature, the RBON maintains an $L^{2}$ relative error of less than $10\%$, with NRBON performing similarly. Figure 3 displays a comparison between the trained networks’ global temperature predictions and actual global temperature readings. The left graph shows results when holding out the most recent two years, while the right graph illustrates the outcome when holding out the most recent five years of data. Figure 4 provides the same visual comparison for local temperatures recorded at the observatory that also publishes the atmospheric $\text{CO}_{2}$ measurements []. Note that temperature data for the local set was only available through 2018.

The significance of this result implies a robust model capable of providing reliable future temperature projections based on various atmospheric $\text{CO}_{2}$ scenarios under different climate responses. This robustness stems from the model’s ability to isolate the impact of $\text{CO}_{2}$ on temperature, as the effects of other contributing elements are learned in the operator approximation. While predicting solely based on $\text{CO}_{2}$ measurements provides a simple example, there is an opportunity to include other contributing factors in the operator input to understand how co-variation among several input variables may affect the output.

Figure: Figure 3: Two (left) and five (right) year average global temperature predictions based on $\text{CO}_{2}$ input. Forecast values are in the shaded region.
Refer to caption: x3.png

**Table 2: Average relative $L^{2}$ test error on temperature data with branch and trunk widths in braces.**
| Network | 2/5 yr | Global temp | Local temp |
| --- | --- | --- | --- |
| RBON | 2 | 0.02 $\{5,12\}$ | 0.07 $\{10,9\}$ |
|  | 5 | 0.02 $\{5,12\}$ | 0.07 $\{10,9\}$ |
| NRBON | 2 | 0.14 $\{27,5\}$ | 0.13 $\{40,2\}$ |
|  | 5 | 0.15 $\{27,5\}$ | 0.13 $\{40,2\}$ |
| F-RBON | 2 | 0.02 $\{2,9\}$ | 0.04 $\{17,10\}$ |
|  | 5 | 0.01 $\{2,12\}$ | 0.13 $\{4,2\}$ |

Figure: Figure 4: Two (left) and five (right) year local temperature predictions based on $\text{CO}_{2}$ input. Forecast values are in the shaded region.
Refer to caption: x5.png

Testing revealed that increasing the width of the branch and trunk networks enhances the model’s flexibility to match highly variable and erratic behavior. However, given highly oscillatory data, the plain RBON can occasionally produce peaks and valleys that deviate too far from the data range when increasing model width. In contrast, the NRBON can increase its network size without generating extreme peaks. Consequently, the smaller RBONs used yield a more stable regression appearance, while the larger NRBON networks produce outputs that attempt to capture more of the random extreme values. This results in a slightly higher error ($\leq 0.17$) for the NRBON, but a shape that more closely resembles the true graph. Figures 5 and 6 display the models’ output over a larger time interval, providing a clearer picture of this phenomenon upon close inspection. This difference is particularly evident in the right graph of Figure 6, where the NRBON attempts to better match the variability in the peaks, while the RBON produces a more steady-state prediction when trained on the smaller set of historical data.

For completeness, we include all results pertaining to learning the operator in the frequency domain, namely the F-RBON. These results are presented in Table 2, and the corresponding plots can be viewed in Section A.1 of the Appendix. It’s worth noting that this dataset does not naturally lend itself to a Fourier transform, and the additional computational work is unnecessary since the representation in the time domain is sufficient.

Figure: Figure 5: Two (left) and five (right) year average global temperature predictions based on $\text{CO}_{2}$ input.
Refer to caption: x7.png

Figure: Figure 6: Two (left) and five (right) year average local temperature predictions based on $\text{CO}_{2}$ input.
Refer to caption: x9.png

## 4 Discussion and Conclusion

The RBON and its variants offer a simple yet powerful network architecture with prediction capabilities that yield errors smaller than the current leading operator network. The network’s compact size provides opportunities for enhanced interpretability and reduced computational load, allowing for exact solutions of network parameters. The majority of variation in different training cycles stems from the location and scale parameters of the RBFs, primarily due to K-means’ tendency to converge on local extrema. A quick solution to this issue is to run the K-means algorithm multiple times. Furthermore, the RBON serves as an excellent tool for scientific computing, where recent advancements have only begun to explore the potential of operator networks in various fields. Finally, the RBON’s ability to train on both real and complex-valued inputs, combined with its other strengths, makes it a promising candidate for applications in signal processing and computer vision tasks.

## Appendix A Appendix

### A.1 Temperature model visuals

This section of the Appendix provides the visual results based on the F-RBON model input for the temperature data discussed in Section 3.2.1 of the paper. Figure 7 displays the average monthly global temperature versus the model prediction for an F-RBON based on $\text{CO}_{2}$ measurement input after training on historical data, omitting the most recent two (left graph) and five (right graph) temperature data. Figure 8 displays the same, but displays the results over a longer time interval to capture the complete overall trend.

The figures 9 and 10 show the F-RBON comparison on the local temperature data on both the shorter time axis in 9 versus the longer axis in 10.

Figure: Figure 7: Two (left) and five (right) year average global temperature predictions based on $\text{CO}_{2}$ input.
Refer to caption: x11.png

Figure: Figure 8: Two (left) and five (right) year average local temperature predictions based on $\text{CO}_{2}$.
Refer to caption: x13.png

Figure: Figure 9: Two (left) and five (right) year average global temperature predictions based on $\text{CO}_{2}$ input.
Refer to caption: x15.png

Figure: Figure 10: Two (left) and five (right) year average local temperature predictions based on $\text{CO}_{2}$.
Refer to caption: x17.png

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 3
presentation: 1
questions: Can you please reinforce the methodological part and make more self-contained and more detailed.
Can you extend the comparisons (if you think it makes sense)?
soundness: 3
strengths: The paper is well written and the topic is interesting though not mainstream.
The experiments are relevant.
summary: This paper deals with operator networks working with radial basis functions.
Time and frequency domain are studied.
A couple of experiments are carried out, mainly on PDEs and TSP.
weaknesses: The methodological description of the method is too short and not detailed enough, especially for unfamiliar readers.
Some theoretical statements are made but the rest of the methodology would not allow re-implmenting the method. Figs 1 and 2 do not carry any information whatsoever; most equations about the network are implicitly embedded in the theorem.
Part of the problem is that the paper describes a variant of NO/ON and the authors focus on the increment, probably for the sake of space, at the expense of making the paper difficult to read for the average reader.
With the amount of provided details, the proposed method seems to be very simple and very close to classical RBFNs with a product on top. The PDE and TSP tasks are not very convincing if they are not compared to other architecture and methods (X-RBON get compared to LNO only).

---

confidence: 2
contribution: 2
initial_rating: 5
presentation: 3
questions: perhaps authors can include the LNO results on CO2 to temperature experiment? 

What is difference between RBON and NRBON? can authors clarify key architectural or mathematical differences between RBON and NRBON, and how these differences impact performance
soundness: 3
strengths: Paper proposes a new operator network based on radial basis functions. Paper is clearly written. The quantitative results of RBON, NRBON, and F-RBON outperform LNO on the wave, burgers, and Euler-bernoulli beam equations.
summary: Paper proposes a radial basis operator network, an operator network entirely represented with radial basis functions. The RBON can learn in both time and frequency domain. Experiments include comparison with Laplace neural operator on wave, burgers, and Euler-bernoulli beam equations. Paper shows RBON and NRBON forecasting global and local temperatures based on atmospheric CO2.
weaknesses: small set of experiments. CO2 to temperature experiment lacks baseline. other factors than CO2 affect temperature, and overall there's a lot of fluctuation in temperature, so it is hard to tell how good predictions are from the RBONs. another dataset with experiments may be helpful.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 1
questions: See weakness.
soundness: 2
strengths: Replacing the DNN in DeepONet with the radial basis neural network sounds an interesting idea, leading to improved numerical results.
summary: This paper presents an extension of DeepONet, where the DNN is replaced with a radial basis neural network, and demonstrates the performance of the extension  through solving some partial differential equations.
weaknesses: 1. The presentation is unclear, making it challenging for readers unfamiliar with DeepONet to follow. For example, it lacks a clear outline (or pseudo) of the training algorithm for the proposed network. What are the training parameters? How should  M and N be selected? 

2. Figure 1: what does `x' represent?  Does the linear transformation L include tunable parameters? 

3. In Section 2.3, the authors mentioned computation in the frequency domain, but the algorithm is not detailed. 

4. The comparison with DeepONet is not included in the paper. 

5. Theoretical contribution of the paper is marginal.

---

confidence: 2
contribution: 3
initial_rating: 3
presentation: 3
questions: 1. Could you elaborate on the literature review regarding learning operators in either the time domain or frequency domain?
2. What are the advantages of learning an operator simultaneously in both the time and frequency domains, as opposed to learning them separately and then combining the results?
3. Could you provide more details on the network structure?
4. How many centers are chosen using KNN, and what criteria are used for selection?
5. How do you explain the results in Table 1, where different networks show instability on different examples? LNO works better on Burgers?
6. Why some networks work better on OOD data in table 1?
soundness: 3
strengths: They introduce the Radial Basis Operator Network (RBON) to learn an operator in both the time and frequency domains, with adjustments to handle complex-valued inputs. Experiments on both in-distribution and out-of-distribution (OOD) data are conducted to validate RBON’s effectiveness.
summary: They introduce the Radial Basis Operator Network (RBON) to learn an operator in both the time and frequency domains, with adjustments to handle complex-valued inputs. Experiments on both in-distribution and out-of-distribution (OOD) data are conducted to validate RBON’s effectiveness.
weaknesses: 1. The experimental comparisons are insufficient, especially lacking comparisons with methods like DeepONet and FNO.
2. Using KNN to select centers may introduce instabilities.
3. The explanation of the experimental results is inadequate; for instance, Table 1 lacks clarity on why different algorithms perform variably across different problems.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: •	Weather data is simple. Can more complex data be added to the comparison as well? There are many time series benchmark data sets (https://arxiv.org/pdf/2303.06053). 
•       More comparison needs to be added. Like LNO, FNO, LSTM, etc. This will give a holistic view wrt performance.
•	Any insights on why the model did not perform as well as global in the local prediction? Is it because of short-term fluctuations or long-term trends?
•	Figures 5 and 6 are too compact and don’t add much information. Can be added to the appendix. A zoomed-in version of the plot would be helpful.
•	A sensitivity study for k means for deciding centers for RBFs would be useful. Showing the effect of varying values of k on model performance will give many insights.
soundness: 3
strengths: The paper is written.
Theory enhances the paper.
summary: This paper introduces a Radial basis operator network for mapping infinite dimensional spaces like function spaces. This approach can model temporal as well as frequency information. The introduction gives a good motivation for the problem, the literature is vast, and the methods are explained well. The paper shows the efficacy of the proposed approach on a weather dataset.
weaknesses: The experiments and comparisons are a bit small and need to be extended to understand the efficacy of RBON.

Meta-review: The paper proposes a novel (radial basis) operator network capable of  learning nonlinear operators in both time and frequency domains, particularly when 
handling complex-valued inputs.
Several critical points (presentation, experiments, baselines,...) have been raised by the reviewers and some of this issue have been
addressed by the authors during the rebuttals and in the revision.  However, there is still some concerns about novelties 
and experimental analysis of the model making it hard to accept the paper in 
its current status.

Decision: Reject