# Fourier Neural Operator for Fluid Flow in Small-Shape 2D Simulated Porous Media Dataset
## 3 Methodology  
### FNO Architecture  
While conducting research on heat propagation, [PERSON] introduced the idea of a harmonic series, later called the Fourier series, which can represent any periodic function as an infinite sum of sine and cosine waves [26]. Assuming \(f(x)\) defined over the interval \((-T,T)\) and outside this space \(f(x+2T)=f(x)\), the Fourier series of this periodic function is written as follows [26]:  
\[f(x)=\frac{a_{0}}{2}+\sum_{n=1}^{\infty}\left[a_{n}\text{cos}(\frac{n\pi x}{T} )+b_{n}\text{sin}(\frac{n\pi x}{T}\ )\right] \tag{6}\]  
in which \(a_{n}\) and \(b_{n}\) are the Fourier series coefficients expressed in the form of integral and also \(a_{0}\) is the first term of \(a_{n}\) when \(n=0\).  
Subsequently, the Fourier transform was developed to extend the Fourier series to non-periodic functions [27]. The Fourier transform involves the decomposition of functions into frequency components. Supposing that \(f(x)\) is the original function, '\(i\)' is the imaginary number (\(\sqrt{-1}\)), and '\(s\)' is the angular frequency, then the mathematical definition of a continuous FT is defined as [27]:  
\[F(s)=\int_{-\infty}^{\infty}f(x)e^{-2\pi isx}dx \tag{7}\]  
It is appropriate to consider the inverse of the continuous FT as [27]:  
\[f(x)=\int_{-\infty}^{\infty}F(s)e^{2\pi isx}ds \tag{8}\]  
The Fourier neural operator is an operator for a neural network that performs convolutions applying the Fourier transform. This causes the higher modes to be removed from the Fourier space, leaving only the lower modes. In the following, a linear transform is applied along with an inverse Fourier transform. This makes the training process independent of the number of cells in a specific mesh.  
At any location \((x,y)\) within a mesh, the FNO algorithm first raises the input \(I(x,y)\) to a higher-dimension channel space \(Z_{0}(x,y)\), where \(Z_{0}(x,y)=P(I(x,y))\) (Figure 1). It does this by locally applying the transform \(P\): \(\mathbb{R}\rightarrow\mathbb{R}^{d_{z}}\), with a parametric procedure using either a Fully Connected (FC) neural network or a simple linear layer. \(Z_{0}\) is defined on the similar mesh to \(I\) and the values of \(Z_{0}\) can be displaced as an image with \(d_{z}\) channels. Then, four successive Fourier layers are applied to \(Z_{0}\). Subsequently, another transform is applied locally \(Q:\ \mathbb{R}^{d_{z}}\rightarrow\ \mathbb{R}\). This final transform projects \(Z_{4}(x,y)\) to the output by \(O(x,y)=Q(Z_{4}(x,y))\). Meantime, \(Z_{4}\) is the output of the fourth (final) Fourier layer and \(Q\) is parameterized by a fully connected neural network.  
\(Z(x,y)\) passes through two routes in the Fourier layers. In the top path, a Fourier transform \(F\), a linear transform \(R\) on the lower Fourier modes, and an inverse Fourier transform \(F^{-1}\) are applied. \(Z(x,y)\) undergoes only a local linear transform \(W\) in the bottom path. Outputs of each path are added together and then subjected to an activation function \(\sigma\) (here ReLu).  
To establish an optimum FNO architecture, PyTorch [28] was employed with Python version 3.9.12. The models developed for the dataset applied _batch size = 5, epochs = 500, step size = 100, gamma = 0.9,_ and _downsampling rate = 1_ in this research. Additionally, 'Adam' [29] was used as the optimizer with a learning rate of 0.001 and a weight decay of 0.0001. Adam uses a distinct learning rate for each scalar parameter and adapts these rates during the whole training process considering the historical values of the partial derivatives of each parameter. This gradient-based algorithm combines the ability of (i) AdaGrad to handle sparse gradients and (ii) RMSProp to function in online and non-stationary settings. The input and output shapes defined were 30 \(\times\) 30 \(\times\) 1 and 30 \(\times\) 30,respectively. Moreover, fast Fourier transform [30] was used as a fast algorithm to compute discrete Fourier transforms and their inverses.