SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Assumption 1. Let  $\beta >2$ ,  $q\in \mathbb{N}$ . There exists  $f_{1},f_{2},\dots ,f_{q}\in W_{2}^{\beta}(\tilde{\mathcal{X}})\cap L^{\infty}(\tilde{\mathcal{X}})$ , such that the density  $p^\star :\tilde{\mathcal{X}}\to \mathbb{R}$  satisfies,  $p^\star (x,t) = \sum_{i = j}^{q}f_j^2 (x,t)$ .
The assumption above is quite general and satisfied by a wide family of probabilities, as discussed in the Proposition 5 of [RC21]. We reiterate the result here below.
Proposition 3 (Proposition 5, [RC21]). The assumption above is satisfied by
(i) any  $p^\star (x,t)$  that is  $\beta$  -times differentiable and strictly positive on  $[-R,R]^d\times [0,R],$
(ii) any exponential model  $p(x,t) = e^{-v(x,t)}$  such that  $v\in W_2^\beta (\tilde{\mathcal{X}})\cap L^\infty (\tilde{\mathcal{X}}),$
(iii) any mixture model of Gaussians or, more generally, of exponential models from (ii),
(iv) any  $p$  that is  $\beta + 2$ -times differentiable on  $[-R, R]^d \times [0, R]$  with a finite set of zeros in  $\tilde{\mathcal{X}}$  and with positive definite Hessian in each zero.
Moreover when  $p^\star$  is  $\beta$ -times differentiable over  $[-R,R]^d \times [0,R]$ , then it belongs to  $W_2^\beta (\tilde{\mathfrak{X}}) \cap L^\infty (\tilde{\mathfrak{X}})$ .
We will need to impose some extra mild assumptions on  $f_{j}$  for  $j \in \{1, \dots, q\}$  to approximate the solution of a fractional Fokker-Planck equation because of involvement of the fractional laplacian operator (non-local operator). We will cite these extra assumptions when we describe the approximation results in section 4.2. Apart from that, we also have a boundedness assumption for the coefficients of the PDE in the domain we consider.
Assumption 2. We have for all  $(x,t)\in \tilde{\mathcal{X}}$ ,  $\mu_{i}(x,t)\leq R_{\mu}$ ,  $\frac{\partial\mu_i(x,t)}{\partial x_i}\leq R_{\mu_p}$ , and  $D_{ij}\leq R_d$  for all  $(i,j)\in \{1,\dots ,d\}$ .
It is also obvious to see that the PSD-based representation of probability density vanishes at infinity.
4.1 Approximating Solution of Fokker-Planck Equation
Let us consider the Fokker-Planck equation given in equation (2) and  $p^{\star}(x,t)$  is the solution of Fokker-Planck equation in equation (2) which satisfies the assumption 1. Hence,  $p^{\star}(x,t)$  satisfies the following,
$$
\frac {\partial p ^ {\star} (x , t)}{\partial t} = - \sum_ {i = 1} ^ {d} \frac {\partial}{\partial x _ {i}} \left(\mu_ {i} (x, t) p ^ {\star} (x, t)\right) + \sum_ {i = 1} ^ {d} \sum_ {j = 1} ^ {d} D _ {i j} \frac {\partial^ {2}}{\partial x _ {i} \partial x _ {j}} p ^ {\star} (x, t).
$$
For now, we are ignoring the initial value as we only are interested in approximation error and we will have a uniform approximation of the solution density  $p^{\star}$  and its derivatives up to the order 2 in the domain.
Given  $m$  base points pair  $(\tilde{x}_i,\tilde{t}_i)\in \tilde{\mathcal{X}}$  for  $i\in \{1,2,\dots ,m\}$  of space and time, let us consider the following approximation model for the solution of FPE,
$$
\tilde {p} (x, t) = \sum_ {i, j} A _ {i j} k _ {X, T} ((x, t), (\tilde {x} _ {i}, \tilde {t} _ {i})) \cdot k _ {X, T} ((x, t), (\tilde {x} _ {j}, \tilde {t} _ {j})) \tag {6}
$$
where  $k_{X,T}$  is the joint kernel across space and time and  $A$  is a positive semi-definite matrix of dimension  $\mathbb{R}^{m\times m}$ .  $k_{X,T}$  can be defined as the product of two Gaussian kernels  $k_{X}$  and  $k_{T}$  which are defined on space and time domain respectively i.e.  $k_{X,T}((x,t),(x',t')) = k_X(x,x')\cdot k_T(t,t')$ . We represent  $\phi_X(x) = k_X(x,\cdot)\in \mathcal{H}_X$ ,  $\phi_T(t) = k_T(t,\cdot)\in \mathcal{H}_T$  and  $\phi_{X,T}(x,t) = \phi_X(x)\otimes \phi_T(t)$ . Here, we assume that  $k_{X}$  and  $k_{T}$  are both Gaussian kernels and hence the RKHS  $\mathcal{H}_X$  and  $\mathcal{H}_T$  correspond to the RKHS of the Gaussian kernel. For simplicity, we keep the kernel bandwidth parameter  $\eta$  the same for  $k_{X}$  and  $k_{T}$ . It is clear that the joint kernel  $k_{X,T}$  is also Gaussian with the kernel bandwidth parameter equal to  $\eta$ . Given  $M = \sum_{i,j}A_{ij}\phi_{X,T}(x_i,t_i)\phi_{X,T}(x_j,t_j)^\top$ ,  $p(x,t)$  can also be represented with function  $f((x,t),A,X,\eta)$  as,
$$
\tilde {p} (x, t) = \underbrace {\phi_ {X , T} (x , t) ^ {\top} M \phi_ {X , T} (x , t)} _ {:= f ((x, t); M, \phi_ {X, T})} = \underbrace {\psi (x , t) ^ {\top} A \psi (x , t)} _ {:= f ((x, t), A, \bar {X}, \eta)}, \tag {7}
$$
where  $\psi (x,t)\in \mathbb{R}^n$ ,  $[\psi (x,t)]_i = k_X(x,\tilde{x}_i)\cdot k_T(t,\tilde{t}_i)$  and  $\tilde{X}\in \mathbb{R}^{m\times d + 1}$  is the concatenated data matrix of space and time for  $m$  pair of samples  $(x_{i},t_{i})$  for  $i\in \{1,\dots ,m\}$ .
We can now easily optimize an associate loss function with the FPE to obtain the unknown PSD matrix  $A$ . Let us assume that there exists a positive semi-definite matrix  $A$  for which the PSD model  $\tilde{p}(x,t)$  approximates the true solution  $p^{\star}(x,t)$  well. In that case, our goal is to obtain an upper bound on the following error objective:
$$
\left\| \frac {\partial \left(p ^ {\star} - \tilde {p}\right) (x , t)}{\partial t} + \sum_ {i = 1} ^ {d} \frac {\partial}{\partial x _ {i}} \left(\mu_ {i} (x, t) \left(p ^ {\star} - \tilde {p}\right) (x, t)\right) - \sum_ {i = 1} ^ {d} \sum_ {j = 1} ^ {d} D _ {i j} \frac {\partial^ {2}}{\partial x _ {i} \partial x _ {j}} \left(p ^ {\star} - \tilde {p}\right) (x, t) \right\| _ {L ^ {2} (\bar {x})}. \tag {8}
$$
Our approximation guarantee works in two steps. In the first step, we find an infinite dimensional positive operator  $M_{\varepsilon} : \mathcal{H}_X \otimes \mathcal{H}_T \to \mathcal{H}_X \otimes \mathcal{H}_T$  such that if we denote  $\hat{p}(x,t) = \phi_{X,T}(x,t)^{\top} M_{\varepsilon} \phi_{X,T}(x,t)$  then for some  $M_{\varepsilon}$ ,  $\hat{p}$  is  $\varepsilon$ -approximation of the true density  $p^{\star}$ . Clearly, for a set of functions  $\tilde{f}_1, \dots, \tilde{f}_q \in \mathcal{H}_X \otimes \mathcal{H}_T$ ,  $M_{\varepsilon} = \sum_{j=1}^{q} \tilde{f}_j \tilde{f}_j^{\top}$  is a positive operator from  $\mathcal{H}_X \otimes \mathcal{H}_T \to \mathcal{H}_X \otimes \mathcal{H}_T$ . Now, we carefully construct  $\tilde{f}_j$  for  $j \in \{1, \dots, q\}$  in RKHS  $\mathcal{H}_X \otimes \mathcal{H}_T$  such that approximation guarantee in theorem 1 is satisfied. Details of the construction is given in Appendix. We represent  $(x,t)$  as a  $d+1$  dimensional vector  $\tilde{x}$  and a mollifier function  $g$  (details are in the appendix) such that  $g_v(x) = v^{-(d+1)} g(\tilde{x}/v)$ . Then, we construct our  $\tilde{f}_j = f_j * g_v$  for  $j \in \{1,2,\dots,q\}$ . We prove the result in Theorem 1 with this construction. To prove our approximation error guarantee for the solution of FPE using PSD model, we assume the derivatives of  $f_1, \dots, f_q$  up to order 2 are bounded which is already covered in Assumption 1.
Theorem 1. Let  $\beta >2,q\in \mathbb{N}$ . Let  $f_{1},\ldots ,f_{q}\in W_{2}^{\beta}(\mathbb{R}^{d})\cap L^{\infty}(\mathbb{R}^{d})$  and the function  $p^\star = \sum_{i = 1}^{q}f_i^2$ . Let  $\varepsilon \in (0,1]$  and let  $\eta \in \mathbb{R}_{++}^{d}$ . Let  $\phi_{X,T}$  be the feature map of the Gaussian kernel with bandwidth  $\eta$  and let  $\mathcal{H}_X\otimes \mathcal{H}_T$  be the associated RKHS. Then there exists  $\mathsf{M}_{\varepsilon}\in \mathbb{S}_{+}(\mathcal{H}_{X}\otimes \mathcal{H}_{T})$  with  $\mathrm{rank}(\mathsf{M}_{\varepsilon})\leq q$ , such that for the representation  $\hat{p} (x,t) = \phi_{X,T}^{\top}\mathsf{M}_{\varepsilon}\phi_{X,T}$ , following holds under assumption 2 on the coefficients of the FPE,
$$
\left\| \frac {\partial \hat {p} (x , t)}{\partial t} + \sum_ {i = 1} ^ {d} \frac {\partial}{\partial x _ {i}} \left(\mu_ {i} (x, t) \hat {p} (x, t)\right) - \sum_ {i = 1} ^ {d} \sum_ {j = 1} ^ {d} D _ {i j} \frac {\partial^ {2}}{\partial x _ {i} \partial x _ {j}} \hat {p} (x, t) \right\| _ {L ^ {2} (\tilde {x})} = O (\varepsilon), \tag {9}
$$
$$
a n d \operatorname {t r} (\mathsf {M} _ {\varepsilon}) \leq \hat {C} | \eta | ^ {1 / 2} \left(1 + \varepsilon^ {\frac {2 \beta}{\beta - 2}} \exp \left(\frac {\tilde {C} ^ {\prime}}{\eta_ {0}} \varepsilon^ {- \frac {2}{\beta - 2}}\right)\right),
$$
where  $|\eta| = \operatorname*{det}(\operatorname{diag}(\eta))$ , and  $\tilde{C}'$  and  $\hat{C}$  depend only on  $\beta, d$ ,  $\|f_i\|_{W_2^\beta(\mathbb{R}^d)}$ ,  $\|f_i\|_{L^\infty(\mathbb{R}^d)}$ ,  $\left\| \frac{\partial f_i}{\partial x_j} \right\|_{L^2(\mathbb{R}^d)}$ , and  $\left\| \frac{\partial^2 f_i}{\partial x_j \partial x_k} \right\|_{L^2(\mathbb{R}^d)}$ .
Even though, from theorem 1, we know that there exists a positive operator  $\mathsf{M}_{\varepsilon}:\mathcal{H}_X\otimes \mathcal{H}_T\to$ $\mathcal{H}_X\otimes \mathcal{H}_T$  such that  $\hat{p} (x,t) = \phi_{X,T}(x,t)^{\top}\mathsf{M}_{\varepsilon}\phi_{X,T}(x,t)$  approximate the solution  $p^{\star}(x,t)$  of FPE from Theorem 1, it is not clear how to compute  $\mathsf{M}_{\varepsilon}$  as we do not have access of  $f_{j}$  for  $j\in \{1,2,\dots ,q\}$  beforehand. Hence, to get the final approximation error bound we need to further approximate  $\hat{p} (x,t)$  with a PSD model of finite dimension  $\tilde{p} (x,t)$  which can be computed if we have the access to a finite number of base points  $(x_{i},t_{i})$  for  $i\in \{1,\dots ,m\}$  in the domain  $\tilde{\mathfrak{X}}$  . Let us now construct a matrix  $A_{m}$  as follows. Consider  $A_{m} = K_{(X,T)(X,T)}^{-1}\tilde{Z} M\tilde{Z}^{\star}K_{(X,T)(X,T)}^{-1}$ , where  $K_{(X,T)(X,T)}$  is a kernel matrix in  $\mathbb{R}^{m\times m}$  such that  $[K_{(X,T)(X,T)}]_{ij} = \phi_{X,T}(x_i,t_i)^{\top}\phi_{X,T}(x_j,t_j)$ . Let us define the operator  $\tilde{Z}:\mathcal{H}_X\otimes \mathcal{H}_T\to \mathbb{R}^m$  such that  $\tilde{Z} u = (\psi (x_1,t_1)^{\top}u,\dots ,\psi (x_m,t_m)^{\top}u)$  and  $\tilde{Z}^{\star}\alpha = \sum_{i = 1}^{m}\psi (x_{i},t_{i})\alpha_{i}$ . We further define a projection operator  $\tilde{P} = \tilde{Z}^{\star}K_{(X,T)(X,T)}^{-1}\tilde{Z}$ . Hence, we now define the approximation density  $\tilde{p} (x,t)$  as
$$
\tilde {p} (x, t) = \underbrace {\psi (x , t) ^ {\top} A _ {m} \psi (x , t)} _ {:= f ((x, t) , A _ {m} , \bar {X} , \eta)} = \underbrace {\phi_ {X , T} (x , t) ^ {\top} \tilde {P} M \tilde {P} \phi_ {X , T} (x , t)} _ {:= f ((x, t) ; \tilde {P} M \tilde {P}, \phi_ {X , T})}.
$$
In the main result of this section below, we show that  $\tilde{p}(x,t)$  is a good approximation for  $\hat{p}(x,t)$  and hence for the true solution  $p^{\star}(x,t)$ .
Theorem 2. Let  $p^{\star}$  satisfy Assumptions 1 and 2. Let  $\varepsilon > 0$  and  $\beta > 2$ . There exists a Gaussian PSD model of dimension  $m \in \mathbb{N}$ , i.e.,  $\tilde{p}(x,t) = f((x,t), A_m, X, \eta)$ , with  $A_m \in \mathbb{S}_+^m$  and  $X \in \mathbb{R}^{m \times d + 1}$ .
and  $\eta_{m}\in \mathbb{R}_{++}^{d}$ , such that with  $m = O(\varepsilon^{-(d + 1) / (\beta -2)}(\log \frac{1}{\varepsilon})^{(d + 1) / 2})$ , we have,
$$
\left\| \frac {\partial \tilde {p} (x , t)}{\partial t} + \sum_ {i = 1} ^ {d} \frac {\partial}{\partial x _ {i}} \left(\mu_ {i} (x, t) \tilde {p} (x, t)\right) - \sum_ {i = 1} ^ {d} \sum_ {j = 1} ^ {d} D _ {i j} \frac {\partial^ {2}}{\partial x _ {i} \partial x _ {j}} \tilde {p} (x, t) \right\| _ {L ^ {2} (\tilde {x})} = O (\varepsilon). \tag {10}
$$
Theorem 2 indicates that there is a positive semi-definite matrix  $A_{m}\in \mathbb{R}^{m\times m}$  such that  $\tilde{p} (x,t) = \psi (x,t)^{\top}A_{m}\psi (x,t)$  is a good approximation of  $p^{\star}(x,t)$ . Next, we would see the usefulness of the above result in Theorem 2 in sampling from the solution of an FPE given the drift term  $\mu$  satisfies the regularity condition as in [BRS16, Theorem 1.1] and  $\mu \in W_2^\beta (\tilde{\mathcal{X}})$ . In this case, the drift function  $\mu$  can be approximated by a function  $\hat{\mu}$  in a Gaussian RKHS, and the approximation guarantees are due to interpolation result in Lemma 4. The following result is the direct consequence of results in the Theorem 2 and Proposition 2.
Efficient sampling method Let  $\hat{\mu} \in \mathcal{H}_{\eta}$  be the approximation of  $\mu$  such that  $\| \mu - \hat{\mu} \|_{L^2 (\tilde{\mathcal{X}})} = O(\varepsilon)$ , by Gaussian kernel approximation. Let us apply the construction of Thm.2 to  $\hat{\mu}$ .
Step 1: Find the best model  $\hat{A}_m$  by minimizing the l.h.s. of equation (12) over the set of  $m$ -dimensional Gaussian PSD models. Note that in that case, since everything is a linear combination of Gaussians (or products of Gaussians), we can compute the  $L^2$  distance in closed form and minimize it over the positive semidefinite cone of  $m$ -dimensional matrices, obtaining  $\hat{A}_{\gamma}$  exactly.
Step 2: Denote the resulting PSD model by  $\hat{p}_{\gamma}(x,t) = f(x,t;\hat{A}_{\gamma},\eta)$ . At any time  $t$  of interest we sample from the probability  $\hat{p}_{\gamma}(\cdot ,t)$  via the sampling algorithm in [MFBR22].
Corollary 1 (Error of sampling). Assume that  $\mu \in W_2^\beta (\tilde{\mathcal{X}})$  satisfies the regularity condition as in [BRS16, Theorem 1.1]. For any time  $t$ , the algorithm described above provides i.i.d. samples from a probability  $\tilde{p}_t$  that satisfies in expectation  $\mathbb{E}_t\mathbb{W}_1(p_t^\star ,\tilde{p}_t)^2 = O(\varepsilon^2)$ , where  $p_t^\star \coloneqq p^\star (\cdot ,t)$  is the probability at time  $t$  of a stochastic process with drift  $\mu$  and diffusion  $D$ . Moreover, the cost of the algorithm is  $O(m^{2}d\log (d / \varepsilon))$  where  $m = O(\varepsilon^{-d / \beta -2}(\log (1 / \varepsilon))^{d})$ .
This result is proven in the appendix after theorem 2. It turns out that the parameter of the PSD model can be learned by a simple semi-definite program [RMFB20] which has a computational complexity of  $O(m^{3.5}\log (1 / \varepsilon))$  where  $m$  is the dimension of the model.
Remark. It is evident from the aforementioned result that our method excels in sampling from the Stochastic Differential Equation (SDE) while ensuring the particle density adheres to Assumption 1 within a bounded domain. In contrast, traditional sampling methods such as Langevin struggle to sample from this extensive class due to the absence of dissipativity, log-sobolev, and Poincaré-like conditions.
4.2 Approximating Solution of Fractional Fokker-Planck Equation
In this section, our aim is to approximate the solution  $p^{\star}(x,t)$  of a fractional Fokker-Planck equation, similar to the previous section. The equation replaces the Laplacian operator with a fractional Laplacian operator while keeping other terms unchanged. The fractional Laplacian operator, being non-local, is generally challenging to compute analytically. Therefore, approximating the fractional operator on a function has become a separate research direction. Below, we demonstrate that utilizing a PSD model-based representation is a natural choice for approximating the fractional operator acting on probability density. We present results for two cases: (i) a Gaussian kernel with the bandwidth parameter  $\eta$ , and (ii) a shift-invariant kernel represented using Bochner's theorem [Rud17].
Theorem 3. Consider probability density represented by a PSD model as  $p(x) = f(x; A, X, \eta) = \sum_{i,j=1}^{m} A_{ij} k(x_i, x) k(x_j, x)$  for some PSD matrix  $A$ , where  $X \in \mathbb{R}^{m \times d}$  is the data matrix whose  $i$ -th row represents sample  $x_i \in \mathbb{R}^d$  for  $i \in \{1, \dots, m\}$ , then
1. if the kernel  $k$  is Gaussian with bandwidth parameter  $\eta$ , we have
$$
\left(- \Delta\right) ^ {s} p (x) = C \sum_ {i, j = 1} ^ {m} A _ {i j} e ^ {- \eta \left(\| x _ {i} \| ^ {2} + \| x _ {j} \| ^ {2}\right)} \mathbb {E} _ {\xi \sim \mathcal {N} (\mu , \Sigma)} \left[ \| \xi \| ^ {2 s} e ^ {- j \xi^ {\top} x} \right]
$$
where  $\mu = 2j\eta (x_i + x_j)$ ,  $\Sigma = 4\eta I$  and  $C$  can be computed in the closed form.
2. if the kernel  $k$  have the bochner's theorem representation, i.e.  $k(x - y) = \int_{\mathbb{R}^d} q(\omega)e^{j\omega^\top (x - y)} d\omega$  where  $q$  is a probability density, then
$$
(- \Delta) ^ {s} p (x) = \sum_ {i, j = 1} ^ {m} A _ {i j} \mathbb {E} [ \| \omega_ {\ell} + \omega_ {k} \| ^ {2 s} e ^ {j \omega_ {\ell} ^ {\top} (x _ {i} + x)} e ^ {j \omega_ {k} ^ {\top} (x _ {j} + x)} ],
$$
where expectations are over  $\omega_{\ell},\omega_{k}\sim q(\cdot)$
We utilize the Fourier multiplier-based definition of the fractional Laplacian operator in Theorem 3. In both cases discussed in Theorem 3, empirical estimation of  $(- \Delta)^s p(x)$ , obtained from finite samples (empirical average), is sufficient for practical purposes. Shifting our focus from the representation of a non-local operator acting on a probability density, we now turn to the approximation result for the solution of a fractional Fokker-Planck equation in this section. We assume that the optimal solution  $p^{\star}(x, t)$  can be expressed as a sum of squares of  $q$  functions, as stated in assumption 1. To establish the approximation results for the fractional Fokker-Planck equation, we need to impose an additional mild assumption on  $f_j$  for  $j \in 1, \dots, q$ , in addition to assumption 1, which we state next.
Assumption 3. Let  $\mathcal{F}[f](\cdot)$  denotes the Fourier transform of a function  $f$ , then  $\mathcal{F}[f_i](\cdot) \in L^1(\tilde{\mathcal{X}}) \cap L^\infty(\tilde{\mathcal{X}})$ , and  $\mathcal{F}[(-\Delta)^s f_i](\cdot) \in L^1(\tilde{\mathcal{X}})$  for  $i \in \{1, \dots, q\}$ .
Similar to the previous section, it requires two steps to obtain the approximation error bound. By similar construction as in section 4.1, we define an infinite dimensional positive operator  $M_{\varepsilon}:\mathcal{H}_X\otimes \mathcal{H}_T\to \mathcal{H}_X\otimes \mathcal{H}_T$  and based on this positive operator, we have a probability density  $\hat{p}$  as  $\hat{p} (x,t) = \phi_{X,T}(x,t)^{\top}M_{\varepsilon}\phi_{X,T}(x,t)$  which is  $O(\varepsilon)$  approximation of  $p^{\star}(x,t)$ . The result is stated below in Theorem 4 and proven in Appendix C.3.
Theorem 4. Let  $\beta >2,q\in \mathbb{N}$ . Let  $f_{1},\ldots ,f_{q}$  satisfy assumptions 1 and 3 and the function  $p^{\star} = \sum_{i = 1}^{q}f_{i}^{2}$ . Let  $\varepsilon \in (0,1]$  and let  $\eta \in \mathbb{R}_{++}^{d}$ . Let  $\phi_{X,T}$  be the feature map of the Gaussian kernel with bandwidth  $\eta$  and let  $\mathcal{H}_X\otimes \mathcal{H}_T$  be the associated RKHS. Then there exists  $\mathsf{M}_{\varepsilon}\in \mathbb{S}_{+}(\mathcal{H}_{X}\otimes \mathcal{H}_{T})$  with  $\mathrm{rank}(\mathsf{M}_{\varepsilon})\leq q$ , such that for the representation  $\hat{p} (x,t) = \phi_{X,T}^{\top}\mathsf{M}_{\varepsilon}\phi_{X,T}$ , following holds under assumption 2 on the coefficients of the fractional FPE,
$$
\left\| \frac {\partial \hat {p} (x , t)}{\partial t} + \sum_ {i = 1} ^ {d} \frac {\partial}{\partial x _ {i}} \left(\mu_ {i} (x, t) \hat {p} (x, t)\right) + (- \Delta) ^ {s} \hat {p} (x, t) \right\| _ {L ^ {2} (\tilde {x})} = O (\varepsilon), \tag {11}
$$
$$
a n d \mathrm {t r} (\mathsf {M} _ {\varepsilon}) \leq \hat {C} | \eta | ^ {1 / 2} \left(1 + \varepsilon^ {\frac {2 \beta}{\beta - 2 s}} \exp \left(\frac {\tilde {C} ^ {\prime}}{\eta_ {0}} \varepsilon^ {- \frac {2}{\beta - 2 s}}\right)\right),
$$
where  $|\eta| = \operatorname{det}(\operatorname{diag}(\eta))$ , and  $\tilde{C}$  and  $\hat{C}$  depend only on  $\beta, d$  and properties of  $f_1, \dots, f_q$ .
Major difficulty in proving the result in Theorem 4 arises from the challenge of effectively managing the fractional Laplacian term. To tackle this issue, we utilize the Fourier-based representation to gain control over the error associated with the fractional Laplacian. The details can be found in Lemma7, presented in Appendix C.1. As a next step, we would follow a similar procedure as in section 4.1 to obtain the final approximation bound. In the second part of the proof, we utilize Gagliardo-Nirenberg inequality [MP18] for fractional Laplacian to obtain the final approximation guarantee. In the next result, we show that for a particular choice of a PSD matrix  $A_{m}$ ,  $\tilde{p}(x,t) = \psi(x,t)^{\top}A_{m}\psi(x,t) = \phi_{X,T}(x,t)^{\top}\tilde{P}M\tilde{P}\phi_{X,T}(x,t)$  is a good approximation for the true solution  $p^{\star}(x,t)$  of the fractional Fokker-Planck equation.
Theorem 5. Let  $p^{\star}$  satisfy Assumptions 1, 2, and 3. Let  $\varepsilon > 0$  and  $\beta > 2$ . There exists a Gaussian PSD model of dimension  $m \in \mathbb{N}$ , i.e.,  $\tilde{p}(x,t) = f((x,t), A_m, X, \phi_{X,T})$ , with  $A_m \in \mathbb{S}_+^m$  and  $X \in \mathbb{R}^{m \times d + 1}$  and  $\eta_m \in \mathbb{R}_{++}^d$ , such that with  $m = O(\varepsilon^{-(d + 1) / (\beta - 2s)} (\log \frac{1}{\varepsilon})^{(d + 1) / 2})$ , we have
$$
\left\| \frac {\partial \tilde {p} (x , t)}{\partial t} + \sum_ {i = 1} ^ {d} \frac {\partial}{\partial x _ {i}} \left(\mu_ {i} (x, t) \tilde {p} (x, t)\right) + (- \Delta) ^ {s} \tilde {p} (x, t) \right\| _ {L ^ {2} (\tilde {\mathcal {X}})} \leq O (\varepsilon). \tag {12}
$$
Similar results as in Corollary 1 can be obtained for the case fractional Fokker-Planck equations as well. We can directly utilize results from [RZGS23] to bound the Wasserstein metric between the solution of two fractional FPE whose drift terms are close in  $L^{\infty}$  metric. However, to find a PSD matrix  $A_{m}$  from finite samples does require two separate sampling procedures which would contribute to the estimation error, (i) sampling to estimate the mean in the estimation of the fractional Laplacian and (ii) sampling of a finite number of data points from the data generating distribution.
Remark. As far as we know, Theorem 5 represents the first proof of approximation error bounds for the solution of the Fractional Fokker-Planck Equation when the approximated solution is a density function. This significant result allows for the utilization of the algorithm presented in [MFBR22] to sample from the approximate solution.
In this paper, we study the approximation properties of PSD models in modeling the solutions of FPEs and fractional FPEs. For the FPE, we show that a PSD model of size  $m = \tilde{O}(\varepsilon^{-(d+1) / (\beta-2)})$  can approximate the true solution density up to order  $\varepsilon$  in  $L^2$  metric where  $\beta$  is the order of smoothness of the true solution. Furthermore, we extend our result to the fractional FPEs to show that the required model size to achieve  $\varepsilon$ -approximation error to the solution density is  $m = \tilde{O}(\varepsilon^{-(d+1) / (\beta-2s)})$  where  $s$  is the fractional order of the fractional Laplacian. In the process, we also show that PSD model-based representations for probability densities allow an easy way to approximate the fractional Laplacian operator (non-local operator) acting on probability density. As a future research direction, we would like to investigate and obtain a finite sample bound on the estimation error under regularity conditions on the drift term  $\mu$ .
Anant Raj is supported by the a Marie Sklodowska-Curie Fellowship (project NN-OVEROPT 101030817). Umut Şimşekli's research is supported by the French government under management of Agence Nationale de la Recherche as part of the "Investissements d'avoir" program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute) and the European Research Council Starting Grant DYNASTY - 101039676. Alessandro Rudi acknowledges support of the French government under management of Agence Nationale de la Recherche as part of the "Investissements d'avir" program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute) and support of the European Research Council (grant REAL 947908).