SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Paths with values in infinite dimensional spaces While we have defined the signature kernel for paths of bounded variation with values in  $\mathbb{R}^{d_x}$ , the kernel is still well-defined when  $\mathbb{R}^{d_x}$  is replaced with a generic Hilbert space  $V$ . Remarkably, even when  $V$  is infinite dimensional, the evaluation of the kernel can be carried out, as Equation (2) only depends on pairwise inner products between the values of the input paths. In particular, the kernel can be evaluated on paths taking their values in functional spaces, which has far-reaching consequences in practice. For example, this gives the flexibility to map the values of finite dimensional input paths into a possibly infinite dimensional feature space, such as the reproducing kernel Hilbert space of a kernel  $\kappa$  on  $\mathbb{R}^{d_x}$ , that is,  $V = \mathcal{H}_{\kappa}$ . This also provides a natural kernel for spatiotemporal signals, such as paths taking their values in  $V = L^2(D)$ , the space of square-integrable functions on a compact domain  $D \subset \mathbb{R}^d$ . For practical applications, the inner product in Equation (2) can be approximated using discrete observations of the input signals on a mesh of the spatial domain  $D$ . The inner product in  $L^2(D)$  can be replaced with more general kernels as those introduced in [WD22]. While it has become common practice to use signature kernels on the RKHS-lifts of Euclidean-valued paths, the ability to define and compute signature kernels on spatiotemporal signals has been, to our knowledge, overlooked in the literature.
We perform experiments across five datasets. First is a univariate synthetic example, the benchmark Black-Scholes model, which permits to readily verify the quality of simulated outputs. The second synthetic example is a state of-the-art univariate stochastic volatility model, called rough Bergomi model. The rough Bergomi model realistically captures many relevant properties of options data, but due to its rough (and hence non-Markovian) nature it is well-known to be difficult to simulate. The third is a multidimensional example with foreign exchange (forex, or FX) currency pairs, which was chosen not only because of the relevance and capitalisation of FX markets but also due to its well-known intricate complexity. Fourth is a univariate example, where we demonstrate the method's ability to condition on relevant variables, given by paths. Finally we present a spatiotemporal generative example, where we seek to simulate the dynamics of the NASDAQ limit order book.
For the unconditional examples, we compare against the SDE-GAN from  $\left[\mathrm{KFL}^{+}21\right]$  and against the same pipeline as the one we proposed, but using an approximation  $\phi_{\mathrm{sig}}^N$  of the signature kernel score  $\phi_{\mathrm{sig}}$  obtained by truncating signatures at some level  $N \in \mathbb{N}$ . We evaluate each training instance with a variety of metrics. The first is the Kolmogorov-Smirnov (KS) test, which is a nonparametric two-sample test between two empirical probability distributions on  $\mathbb{R}$ , see [Smi39] for more details. We apply the KS test on the marginals between a batch of generated paths against an unseen batch from the real data distribution. We repeated this test 5000 times at the  $5\%$  significance level and reported the average KS score along with the average Type I error. Each training instance was kept to a maximum of 2 hours for the synthetic examples, and 4 hours for the real data example. Finally, as mentioned at the end of Section 3.5, when training with respect to  $\phi_{\mathrm{sig}}$  we mapped path state values into  $(\mathcal{H}, \kappa)$  where  $\kappa$  denotes the RBF kernel on  $\mathbb{R}^d$ . Additional details on hyperparameter selection, learning rates, optimisers and further evaluation metrics can be found in the Appendix.
4.1 Geometric Brownian motion
As a toy example, we seek to learn a geometric Brownian motion (gBm) of the form
$$
d y _ {t} = \mu y _ {t} d t + \sigma y _ {t} d W _ {t}, \quad y _ {0} = 1, \tag {9}
$$
We chose  $\mu = 0, \sigma = 0.2$  and generated time-augmented paths of length 64 over the grid  $\Delta = \{0,1,\ldots,63\}$  with  $dt = 0.01$ . Thus our dataset is given by time-augmented paths  $y:[0,63] \to \mathbb{R}^2$  embedded in path space via linear interpolation. For all three discriminators, the training and test set were both comprised of 32768 paths and the batch size was chosen to be  $N = 128$ . We trained the SDE-GAN for 5000 steps,  $\phi_{\mathrm{sig}}$  for 4000 and  $\phi_{\mathrm{sig}}^{N}$  for 10000 steps. Table 1 gives the KS scores along
each of the specified marginals, along with the percentage Type I error. Here the generator trained with  $\phi_{\mathrm{sig}}$  performs the best, achieving a Type I error at the assumed confidence level.
Model	t=6	t=19	t=32	t=44	t=57
SDE-GAN	0.1641, 41.1%	0.1094, 5.2%	0.1421, 24.2%	0.1104, 5.9%	0.1427, 26.2%
φsigN(N=3)	0.1298, 15.4%	0.1277, 16.1%	0.1536, 37.4%	0.2101, 78.8%	0.2416, 92.3%
φsig(ours)	0.1071, 5.0%	0.1084, 6.0%	0.1086, 5.9%	0.1089, 5.8%	0.1075, 5.5%
4.2 Rough Bergomi volatility model
It is well-known that the benchmark model (9) oversimplifies market reality. More complex models, (rough) stochastic volatility (SV) were introduced in the past decades, that are able to capture relevant properties of market data are used by financial practitioners to price and hedge derivatives. Prominent examples of stochastic volatility mdoels include the Heston and SABR models [HKLW02, HLW15, Hes93]. State-of-the-art models in this context have been introduced in [GJR18]. They display a stochastic volatility with rough sample paths. Most notable among these for pricing and hedging is the rough Bergomi (rBergomi) model [BFG16] which is of the form
$$
d y _ {t} = - \frac {1}{2} V _ {t} d t + \sqrt {V _ {t}} d W _ {t} \quad \text {w h e r e} \quad d \xi_ {t} ^ {u} = \xi_ {t} ^ {u} \eta \sqrt {2 \alpha + 1} (u - t) ^ {\alpha} d B _ {t}, \tag {10}
$$
and where  $\xi_t^u$  is the instantaneous forward variance for time  $u$  at time  $t$ , with  $\xi_t^t = V_t$ , and  $\alpha = H - 1/2$  where  $H$  is the Hurst exponent. The parameter set is given by  $(\eta, \rho, H)$  with initial conditions  $X_0 = x$  and  $\xi_t^u = \xi_0$ . It has been a well-known headache for modellers that—despite their many modelling advantages—rough volatility models (such as (10)) are slow to simulate with traditional methods. We demonstrate how our method can be used to capture the dynamics of the rough Bergomi model (10), and in passing we also note that our method provides a significant simulation speedup for (10) compared to previously available simulation methods.
Table 1: KS test average scores and Type I errors on marginals on gBm.
Model	t=6	t=19	t=32	t=44	t=57
SDE-GAN	0.1929, 68.3%	0.2244, 86.2%	0.2273, 87.0%	0.2205, 83.4%	0.1949, 68.7%
φsigN(N=5)	0.1126, 8.1%	0.1172, 10.1%	0.1146, 8.2%	0.1153, 8.5%	0.1134, 7.0%
φsig(ours)	0.1086, 5.4%	0.1129, 5.9%	0.1118, 5.2%	0.1127, 6.2%	0.1159, 6.9%
Table 2: KS test average scores and Type I errors on marginals on rBergomi model
To do so, we simulate paths of length 64 over the time window to  $[0,2]$ , and specify  $dt = 1 / 32$ . Thus paths are of length 64. The parameters are  $(\xi_0,\eta ,\rho ,H) = (0.04,1.5, - 0.7,0.2)$  and set  $d = 1$ . Paths are again time-augmented. The hyperparameters for training are the same as in the previous section. The results on the marginal distributions are summarized in Table 2. We see that training with respect to  $\phi_{\mathrm{sig}}$  vastly outperforms the other two discriminators.
4.3 Foreign exchange currency pairs
We consider an example where samples from the data measure  $\mathbb{P}_{X^{\mathrm{true}}}$  are time-augmented paths  $y:[0,T]\to \mathbb{R}^3$  corresponding to hourly market close prices of the currency pairs EUR/USD and USD/JPY $^4$ . To deal with irregular sampling, we linearly interpolate each sample  $y$  over a fixed grid  $\Delta = \{t_0,t_1,\ldots ,t_{63}\}$ . Training hyperparameters were kept the same as per the rBergomi example: paths are comprised of 64 observations, the batch size was taken to be  $N = 128$ , and the number of training epochs was taken to be 10000 for the SDE-GAN, 4000 for  $\phi_{\mathrm{sig}}$  and 15000 for  $\phi_{\mathrm{sig}}^{N}$ . KS scores for each of the marginals are given in Table 3 and 4. We note that only the generator trained with  $\phi_{\mathrm{sig}}$  is able to achieve strong performance on nearly all marginals.
4Data is obtained from https://www.dukascopy.com/swiss/english/home/
Model	t=6	t=19	t=32	t=44	t=57
SDE-GAN	0.1889, 62.9%	0.2760, 98.2%	0.3324, 99.9%	0.3781, 100.0%	0.4209, 100.0%
φsigN(N=5)	0.1098, 4.2%	0.1279, 12.0%	0.1399, 18.7%	0.1507, 28.1%	0.1608, 37.5%
φsig(ours)	0.1270, 12.8%	0.1085, 5.2%	0.1060, 4.3%	0.1065, 5.1%	0.1049, 4.0%
Table 3: KS test average scores on marginals (EUR/USD)
Model	t=6	t=19	t=32	t=44	t=57
SDE-GAN	0.1404, 20.5%	0.1665, 44.2%	0.1771, 56.4%	0.1855, 63.8%	0.1948, 70.3%
φsigN(N=5)	0.1666, 43.8%	0.1877, 72.4%	0.2008, 84.7%	0.2154, 93.2%	0.2311, 98.3%
φsig(ours)	0.1189, 9.2%	0.1121, 5.8%	0.1069, 4.9%	0.1075, 3.8%	0.1051, 3.3%
Table 4: KS test average scores on marginals (USD/JPY).
We also present a histogram of sample correlations between generated EUR/USD and USD/JPY paths for each of the three discriminators alongside those from the data distribution. From Figure 1 it appears that only the Neural SDE trained with  $\phi_{\mathrm{sig}}$  correctly identifies the negative correlative structure between the two pairs. This is likely due to the fact that these dependencies are encoded in higher order terms of the signature that the truncated method does not capture.
Figure 1: Histogram of correlation coefficients between EURUSD and USDJPY pairs, 1024 samples.
We now consider a conditional generation problem, where the conditioning variables are time-augmented paths  $\mathbb{Q} \sim x: [t_0 - dt, t_0] \to \mathbb{R}^2$  representing the trajectory of prior  $dt = 32$  observations of EUR/USD 15-minute close prices, and the target distribution is  $X^{\mathrm{true}}: [t_0, t_0 + dt'] \to \mathbb{R}^2$  representing the following  $dt' = 16$  observations. Given batched samples  $\{x^i, y^i\}_{i=1}^N$ , where  $x^i \sim \mathbb{Q}$  and  $y^i \sim \mathbb{P}_{X^{\mathrm{true}}}(\cdot | x^i)$ , we train our generator according to equation (7). We encoded the conditioning paths via the truncated (log) signatures of order 5, and fed these values into each of the neural networks of the Neural SDE. In Figure 2, it is evident that the conditional generator exhibits the capability to produce conditional distributions that frequently encompass the observed path. Furthermore, it is noteworthy that these generated distributions capture certain distinctive characteristics of financial markets, such as martingality, mean reversion, or leverage effects when applicable.
4.4 Simulation of limit order books
Here, we consider the task of simulating the dynamics of a limit order book (LOB), that is, an electronic record of all the outstanding orders for a financial asset, representing its supply and demand over time. Simulating LOB dynamics is an important challenge in quantitative finance and several synthetic market generators have been proposed [LWL  $^{+}20$ ],[VBP  $^{+}20$ ],[SCC21],[CPC  $^{+}21$ ],[CMVB22]. An order  $o = (t_{o},x_{o},v_{o})$  submitted at time  $t_{o}$  with price  $x_{o}$  and size  $v_{o} > 0$  (resp.,  $v_{o} < 0$ ) is a commitment to sell (resp., buy) up to  $|v_{o}|$  units of the traded asset at a price no less (resp., no greater)
Figure 2: Given a conditioning path  $x \sim \mathbb{Q}$ , the generator provides (in blue) the conditional distribution  $\mathbb{P}_{X^{\theta}}(\cdot |x)$ . The dotted line gives the true path  $y \sim \mathbb{P}_{X^{\mathrm{true}}}(\cdot |x)$ .
than  $x_{o}$ . Various events are tracked (e.g. new orders, executions, and cancellations) and the LOB  $\mathcal{B}(t)$  is the set of all active orders in a market at time  $t$ . While prior work typically fit a generator that produces the next event, and run it iteratively to generate a sequence of events, we propose to model directly the spatiotemporal process  $Y_{t}(x) = \sum_{o\in \mathcal{B}(t):x_{o} = x}v_{o}$ . To generate LOB trajectories, we use the Neural SPDE model and train it by minimising expected spatiotemporal kernel scores constructed by composing the signature kernel  $k_{\mathrm{sig}}$  with 3 different SE-T type kernels introduced in [WD22], namely the ID, SQR and CEXP kernels. We fit our model on real LOB data from the NASDAQ public exchange  $[\mathrm{NMK}^{+}18]$  which consists of about  $4M$  timestamped events with  $L = 10$  price levels. We split this LOB trace into sub-traces of size  $T = 30$  to construct our dataset. On Figure 3 report the average KS scores for each of the  $L\times T$  marginals, using the 3 different kernel scores.
Figure 3: KS test average scores for each spatiotemporal marginal, 100 runs, NASDAQ data.
This work showcases the utilization of Neural SDEs as a generative model, highlighting their advantages over competitor models in terms of simplicity and stability, particularly via non-adversarial training. Additionally, we show how Neural SDEs exhibit the ability to be conditioned on diverse and intricate data structures, surpassing the capabilities of existing competitor works. We have achieved this by introducing the signature kernel score on paths and by showing their applicability to our setting (by proving strict properness). Performance of our methods are given computational time and memory is competitive with state-of-the-art methods. Moreover, we have shown that this approach extends to the generation of spatiotemporal signals, which has multiple applications in finance including limit order data generation. Further extensions of this work may include extending its generality to include jump processes in the driving noise of the approximator process (Neural SDEs) used. On the theoretical level extensions may include the validity of results to paths with lower regularity than currently considered. Although sample paths from a Stratonovich SDE are not of bounded variation almost surely, sample paths generated by an SDE solver, once interpolated, are piecewise linear, and hence of bounded variation. A similar point can be made about compactness of the support of the measures. It is possible to ensure characteristicness of the signature kernel on non-compact sets of less regular paths using limiting arguments and changing the underlying
topology on pathspace. Further extensions for practical applications can (and should) include the inclusion of more varied evaluation metrics and processes. Notably, in a later step, the generated data should be tested by assessing whether existing risk management frameworks and investment engines can be improved when data used for backtesting is augmented with synthetic samples provided by our methods. Furthermore, the spatiotemporal results can be extended to more complex structures, including being used for the synthetic generation of implied volatility surface dynamics, which has been a notoriously difficult modelling problem in past decades.
ML was supported by the EPSRC grant EP/S026347/1. ZI was supported by EPSRC grant EP/R513064/1.
[ACB17] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pages 214-223. PMLR, 2017.
[ASS20] Imanol Perez Arribas, Christopher Salvi, and Lukasz Szpruch. Sig-sdes model for quantitative finance. In ACM International Conference on AI in Finance, 2020.
[BCR84] Christian Berg, Jens Peter Reus Christensen, and Paul Ressel. Harmonic analysis on semigroups: theory of positive definite and related functions, volume 100. Springer, 1984.
[BFG16] Christian Bayer, Peter Friz, and Jim Gatheral. Pricing under rough volatility. Quantitative Finance, 16(6):887-904, 2016.
[BMN16] Diane Bouchacourt, Pawan K Mudigonda, and Sebastian Nowozin. Disco nets: Dissimilarity coefficients networks. Advances in Neural Information Processing Systems, 29, 2016.
[BO21] Patric Bonnier and Harald Oberhauser. Proper scoring rules, gradients, divergences, and entropies for paths and time series. arXiv preprint arXiv:2111.06314, 2021.
[CFC+21] Thomas Cochrane, Peter Foster, Varun Chhabra, Maud Lemercier, Christopher Salvi, and Terry Lyons. Sk-tree: a systematic malware detection algorithm on streaming trees via the signature kernel. arXiv preprint arXiv:2102.07904, 2021.
[CJB23] Vedant Choudhary, Sebastian Jaimungal, and Maxime Bergeron. Funvol: A multi-asset implied volatility market simulator using functional principal components and neural sdes. arXiv preprint arXiv:2303.00859, 2023.
[CLS23] Nicola Muca Cirone, Maud Lemercier, and Christopher Salvi. Neural signature kernels as infinite-width-depth-limits of controlled resnets. arXiv preprint arXiv:2303.17671, 2023.
[CLX21] Thomas Cass, Terry Lyons, and Xingcheng Xu. General signature kernels, 2021.
[CM21] Rama Cont and Marvin S Müller. A stochastic partial differential equation model for limit order book dynamics. SIAM Journal on Financial Mathematics, 12(2):744-787, 2021.
[CMVB22] Andrea Coletta, Aymeric Moulin, Svitlana Vyetrenko, and Tucker Balch. Learning to simulate realistic limit order book markets from data as a world agent. In Proceedings of the Third ACM International Conference on AI in Finance, pages 428-436, 2022.
$\left[\mathrm{CPC}^{+}21\right]$  Andrea Coletta, Matteo Prata, Michele Conti, Emanuele Mercanti, Novella Bartolini, Aymeric Moulin, Svitlana Vyetrenko, and Tucker Balch. Towards realistic market simulations: a generative adversarial networks approach. In Proceedings of the Second ACM International Conference on AI in Finance, pages 1-9, 2021.
[CT22] Thomas Cass and William F Turner. Topologies on unparameterised path space. arXiv preprint arXiv:2206.11153, 2022.
[PADD21] Lorenzo Pacchiardi, Rilwan Adewoyin, Peter Dueben, and Ritabrata Dutta. Probabilistic forecasting with conditional generative networks via scoring rule minimization. arXiv preprint arXiv:2112.08217, 2021.
[PD22] Lorenzo Pacchiardi and Ritabrata Dutta. Likelihood-free inference with generative neural networks via scoring rule minimization. arXiv preprint arXiv:2205.15784, 2022.
[SCC21] Zijian Shi, Yu Chen, and John Cartlidge. The lob recreation model: Predicting the limit order book from taq history using an ordinary differential equation recurrent neural network. Proceedings of the AAAI Conference on Artificial Intelligence, 35(1):548-556, 2021.
[SCF+21] Christopher Salvi, Thomas Cass, James Foster, Terry Lyons, and Weixin Yang. The signature kernel is the solution of a goursat pde. SIAM Journal on Mathematics of Data Science, 3(3):873-899, 2021.
[SGS18] Carl-Johann Simon-Gabriel and Bernhard Scholkopf. Kernel distribution embeddings: Universal kernels, characteristic kernels and kernel metrics on distributions. The Journal of Machine Learning Research, 19(1):1708-1736, 2018.
[SLG22] Christopher Salvi, Maud Lemercier, and Andris Gerasimovics. Neural stochastic pdes: Resolution-invariant learning of continuous spatiotemporal dynamics. Advances in Neural Information Processing Systems, 35:1333-1344, 2022.
[SLL+21] Christopher Salvi, Maud Lemercier, Chong Liu, Blanka Hovarth, Theodoros Damoulas, and Terry Lyons. Higher order kernel mean embeddings to capture filtrations of stochastic processes. arXiv preprint arXiv:2109.03582, 2021.
[Smi39] Nikolai V Smirnov. On the estimation of the discrepancy between empirical curves of distribution for two independent samples. Bull. Math. Univ. Moscou, 2(2):3-14, 1939.
[SSDK+20] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.
[TR19] Belinda Tzen and Maxim Raginsky. Neural stochastic differential equations: Deep latent gaussian models in the diffusion limit. arXiv preprint arXiv:1905.09883, 2019.
$\left[\mathrm{VBP}^{+}20\right]$  Svitlana Vyetrenko, David Byrd, Nick Petosa, Mahmoud Mahfouz, Danial Dervovic, Manuela Veloso, and Tucker Balch. Get real: Realism metrics for robust limit order book market simulations. In Proceedings of the First ACM International Conference on AI in Finance, pages 1-8, 2020.
[VKK21] Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. Advances in Neural Information Processing Systems, 34:11287-11302, 2021.
[WD22] George Wynne and Andrew B Duncan. A kernel two-sample test for functional data. Journal of Machine Learning Research, 23(73):1-51, 2022.
[YJVdS19] Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. Time-series generative adversarial networks. Advances in neural information processing systems, 32, 2019.
A Signature Kernel Scores
Proof of Proposition 3.3
Proof (Appendix). The general result was first shown in [GR07]. We first show that  $\phi_{\mathrm{sig}}$  is proper. By Proposition 3.1 the signature kernel is positive definite and characteristic on  $\mathcal{P}(\mathcal{K})$ . It remains to show that  $\mathbb{E}_{y\sim \mathbb{Q}}[\phi_{\mathrm{sig}}(\mathbb{Q},y)]\leq \mathbb{E}_{y\sim \mathbb{Q}}[\phi_{\mathrm{sig}}(\mathbb{P},y)]$ . This means we must have
$$
\mathbb {E} _ {x \sim \mathbb {P}, y \sim \mathbb {Q}} [ k _ {\mathrm {s i g}} (x, y) ] \leq \frac {1}{2} \mathbb {E} _ {x, x ^ {\prime} \sim \mathbb {P}} [ k _ {\mathrm {s i g}} (x, x ^ {\prime}) ] + \frac {1}{2} \mathbb {E} _ {y, y ^ {\prime} \sim \mathbb {Q}} [ k _ {\mathrm {s i g}} (y, y ^ {\prime}) ].
$$
Writing  $\mathbb{M} = \frac{1}{2}\mathbb{P} + \frac{1}{2}\mathbb{Q}$ , a modification of Theorem 2.1 in Berg et al. [BCR84] (pg. 235) gives that
$$
\int k _ {\mathrm {s i g}} (x, y) d (\mathbb {P} \otimes \mathbb {Q}) (x, y) \leq \int k _ {\mathrm {s i g}} (x, y) d (\mathbb {M} \otimes \mathbb {M}) (x, y), \tag {11}
$$
where  $\mathbb{P} \otimes \mathbb{Q}$  denotes the natural product measure on  $\mathcal{K} \times \mathcal{K}$ . Re-arranging (11), one arrives at the desired result.
To show strict properness, we need to show that  $\mathbb{E}_{y\sim \mathbb{Q}}[\phi_{\mathrm{sig}}(\mathbb{Q},y)]\leq \mathbb{E}_{y\sim \mathbb{Q}}[\phi_{\mathrm{sig}}(\mathbb{P},y)]$  holds with equality iff  $\mathbb{P} = \mathbb{Q}$  for all  $\mathbb{P},\mathbb{Q}\in \mathcal{P}(\mathcal{K})$ . Suppose that there exists another  $\mathbb{P}'\in \mathcal{P}(\mathcal{K})$  such that  $\mathbb{E}_{y\sim \mathbb{Q}}[\phi_{\mathrm{sig}}(\mathbb{Q},y)] = \mathbb{E}_{y\sim \mathbb{Q}}[\phi_{\mathrm{sig}}(\mathbb{P}',y)]$ . Then we would have that
$$
\mathbb {E} _ {x, x ^ {\prime} \sim \mathbb {P} ^ {\prime}} [ k _ {\mathrm {s i g}} (x, x ^ {\prime}) ] - 2 \mathbb {E} _ {x \sim \mathbb {P} ^ {\prime}, y \sim \mathbb {Q}} [ k _ {\mathrm {s i g}} (x, y) ] + \mathbb {E} _ {y, y ^ {\prime} \sim \mathbb {Q}} [ k _ {\mathrm {s i g}} (y, y ^ {\prime}) ] = 0,
$$
or that  $\mathcal{D}_{k_{\mathrm{sig}}}(\mathbb{P}',\mathbb{Q}) = 0$  , which is only true if  $\mathbb{P}' = \mathbb{Q}$  due to characteristicness of the kernel  $k_{\mathrm{sig}}$
Proof of Proposition 3.4
Proof (Appendix). The proof follows directly from  $\mathrm{[GBR^{+}12]}$ , Lemma 6. Note that an unbiased estimator for  $\mathbb{E}_{x,x^{\prime}\sim \mathbb{P}}[k_{\mathrm{sig}}(x,x^{\prime})]$  from i.i.d samples  $(x_{1},\ldots ,x_{m}),x_{i}\sim \mathbb{P}$  is given by the U-statistic
$$
T _ {U} ^ {1} (x _ {1}, \dots , x _ {m}) = \frac {1}{m (m - 1)} \sum_ {i \neq j} k _ {\mathrm {s i g}} (x _ {i}, x _ {j}).
$$
Moreover, an unbiased estimate of  $\mathbb{E}_{x\sim \mathbb{P}}[k_{\mathrm{sig}}(x,y)]$  is given by
$$
T _ {U} ^ {2} (x _ {1}, \dots , x _ {m}, y) = \frac {1}{m} \sum_ {i = 1} ^ {m} k _ {\mathrm {s i g}} (x _ {i}, y).
$$
Writing  $\hat{\phi}_{\mathrm{sig}}(\mathbb{P},y) = T_U^1 (x_1,\ldots ,x_m) - 2T_U^2 (x_1,\ldots ,x_m,y)$  completes the proof.
All experiments were run on a NVIDIA GeForce RTX 3070 Ti GPU, except the experiment in Section 4.4 for which the NSPDE model was trained using a NVIDIA A100 40GB GPU.