SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We report here details and additional results for the Bayesian optimization experiments from Section 5.2 of the main text. We evaluated our proposed approach on a common synthetic optimization test function, the Hartmann function in its three and six dimensional versions [40, p.185]. Each of two test functions is evaluated on a  $d_x$ -dimensional hypercube  $\mathcal{X} = [0,1]^{d_x}$ .
E.1 Models
We compared our proposals, RCNP and RGNP, against CNP and GNP as well as their attentive variants ACNP and AGNP. A GP with a Matérn- $\frac{5}{2}$  kernel additionally served as the "gold standard" to be compared against. We mostly followed the architectural choices discussed in Section D.1. However, we kept the number of hidden units per layer fixed at 256 and the relational encoding dimension at 128, irrespective of the dimensionality  $d_x$ . The models were trained with a learning rate of  $3 \cdot 10^{-4}$  over up to 500 epochs with  $2^{14}$  datasets in each epoch. A validation set with  $2^{12}$  datasets was used to track the training progress and early stopping was performed if the validation score had not improved for 150 epochs.
<sup>1</sup>See https://www.sfu.ca/~ssurjano/optimization.html for their precise definitions and minima.
Table S6: Synthetic (sawtooth). Comparison of the interpolation (INT) and out-of-input-distribution (OOID) performance of our RCNP models with different CNP baselines on sawtooth function with varying input dimensions. We use normalized log-likelihoods as our metric and show mean and (standard deviation) obtained from 10 runs with different seeds. Trivial refers to a model that predicts a Gaussian distribution utilizing the empirical mean and standard deviation derived from the context outputs. "F" denotes failed attempts that yielded very bad results. Missing entries could not be run. Statistically significantly best results are bolded.
\(d_x=1\)	\(d_x=2\)	\(d_x=3\)	\(d_x=5\)	\(d_x=10\)
INT	OOID	INT	OOID	INT	OOID	INT	OOID	INT	OOID
RCNP (sta)	3.03 (0.06)	3.04 (0.06)	1.73 (0.03)	1.74 (0.03)	0.85 (0.01)	0.85 (0.01)	0.44 (0.00)	0.44 (0.00)	0.75 (0.00)	0.75 (0.00)
RGNP (sta)	3.90 (0.09)	3.90 (0.10)	2.13 (0.33)	2.13 (0.32)	1.09 (0.01)	1.09 (0.01)	1.13 (0.05)	1.13 (0.05)	1.33 (0.03)	1.32 (0.03)
ConvCNP	3.64 (0.04)	3.64 (0.04)	3.66 (0.04)	3.66 (0.04)	-	-	-	-	-	-
ConvGNP	3.94 (0.11)	3.97 (0.08)	4.11 (0.03)	3.99 (0.11)	-	-	-	-	-	-
CNP	2.25 (0.02)	F	1.15 (0.45)	-3.27 (4.72)	0.36 (0.28)	-0.37 (0.12)	-0.03 (0.10)	-0.22 (0.03)	0.27 (0.00)	-2.29 (0.67)
GNP	0.83 (0.04)	F	1.04 (0.09)	F	0.23 (0.13)	F	0.02 (0.05)	F	0.03 (0.04)	F
Trivial	-0.27 (0.00)	-0.27 (0.00)	-0.19 (0.00)	-0.19 (0.00)	-0.19 (0.00)	-0.19 (0.00)	-0.18 (0.00)	-0.18 (0.00)	-0.14 (0.00)	-0.14 (0.00)
Table S7: Synthetic (mixture). Comparison of the interpolation (INT) and out-of-input-distribution (OOID) performance of our RCNP models with different CNP baselines on mixture function with varying input dimensions. We use normalized log-likelihoods as our metric and show mean and (standard deviation) obtained from 10 runs with different seeds. Trivial refers to a model that predicts a Gaussian distribution utilizing the empirical mean and standard deviation derived from the context outputs. "F" denotes failed attempts that yielded very bad results. Missing entries could not be run. Statistically significantly best results are bolded.
dx=1	dx=2	dx=3	dx=5	dx=10
INT	OOID	INT	OOID	INT	OOID	INT	OOID	INT	OOID
RCNP (sta)	0.20 (0.01)	0.20 (0.01)	0.17 (0.00)	0.17 (0.00)	-0.10 (0.00)	-0.10 (0.00)	-0.31 (0.03)	-0.31 (0.03)	-0.32 (0.00)	-0.32 (0.00)
RGNP (sta)	0.34 (0.03)	0.34 (0.03)	0.46 (0.02)	0.46 (0.02)	0.37 (0.01)	0.37 (0.01)	0.04 (0.02)	0.04 (0.02)	-0.11 (0.02)	-0.11 (0.02)
ConvCNP	0.38 (0.02)	0.38 (0.02)	0.63 (0.01)	0.63 (0.01)	-	-	-	-	-	-
ConvGNP	0.49 (0.15)	0.49 (0.15)	0.87 (0.03)	0.87 (0.03)	-	-	-	-	-	-
CNP	0.01 (0.01)	F	-0.22 (0.01)	-7.16 (3.61)	-0.57 (0.11)	-2.55 (1.15)	-0.72 (0.08)	-1.71 (0.55)	-0.88 (0.00)	-1.15 (0.06)
GNP	0.17 (0.01)	F	-0.01 (0.01)	F	-0.17 (0.00)	-0.67 (0.05)	-0.32 (0.00)	-0.72 (0.03)	-0.38 (0.10)	-0.75 (0.09)
Trivial	-0.78 (0.00)	-0.78 (0.00)	-0.81 (0.00)	-0.81 (0.00)	-0.84 (0.00)	-0.84 (0.00)	-0.86 (0.00)	-0.86 (0.00)	-0.87 (0.00)	-0.87 (0.00)
E.2 Data
The datasets used to train the neural process models were generated based on synthetic functions sampled from a Gaussian process model with kernel  $k$ , with context and target set sizes as described in Section D.2. However, while the datasets used in the synthetic regression experiments were generated using fixed kernel setups, the current experiment explored sampling from a set of base kernels. To explore four different training regimes for each  $d_{x} = \{3,6\}$ , we changed how the GP kernel  $k(z) = \theta k_0(z / \ell)$  with an output scale  $\theta$  and a lengthscale  $\ell$  was chosen as follows.
(i) Matérn-fixed. The kernel  $k$  is a Matérn- $\frac{5}{2}$  kernel with fixed  $\ell = \sqrt{d_x} / 4$  and  $\theta = 1$ .
(ii) Matérn-sampled. The kernel  $k$  is a Matérn- $\frac{5}{2}$  kernel, with
$$
\ell \sim \mathcal {L N} \left(\log \left(\sqrt {d _ {x}} / 4\right), 0. 5 ^ {2}\right), \quad \theta \sim \mathcal {L N} (0, 1),
$$
where  $\mathcal{LN}$  is a log-Normal distribution, i.e.,  $\log (\ell)$  follows a standard-Normal distribution.
(iii) Kernel-single. The kernel  $k$  is sampled as
$$
\begin{array}{l} k _ {0} \sim \mathcal {U} \left(\left\{\mathrm {E Q}, \text {M a t e r n} - \frac {1}{2}, \text {M a t e r n} - \frac {3}{2}, \text {M a t e r n} - \frac {5}{2} \right\}\right), \\ \ell \sim \mathcal {L N} \left(\log \left(\sqrt {d _ {x}} / 4\right), 0. 5 ^ {2}\right), \\ \theta \sim \mathcal {L N} (0, 1), \\ \end{array}
$$
where  $\mathcal{U}$  is a uniform distribution over the set.
(iv) Kernel-multiple. The kernel  $k$  is sampled as
$$
\begin{array}{l} k _ {1}, k _ {2}, k _ {3}, k _ {4} \sim \mathcal {U} \left(\left\{\mathrm {N A}, \mathrm {E Q}, \text {M a t e r n} - \frac {1}{2}, \text {M a t e r n} - \frac {3}{2}, \text {M a t e r n} - \frac {5}{2} \right\}\right), \\ \ell_ {i} \sim \mathcal {L N} \left(\log (\sqrt {d _ {x}} / 4), 0. 5 ^ {2}\right), \qquad i \in \{1, \dots 4 \}, \\ \theta_ {j} \sim \mathcal {L N} (0, 1), \qquad j \in \{1, 2 \}, \\ k = \theta_ {1} k _ {1} (\ell_ {1}) \cdot k _ {2} (\ell_ {2}) + \theta_ {2} k _ {3} (\ell_ {3}) \cdot k _ {4} (\ell_ {4}), \\ \end{array}
$$
Table S8: An ablation study conducted on RCNPs and FullRCNPs using synthetic regression data. We evaluated both the 'stationary' (sta) version, which employs the difference comparison function, and the 'isotropic' (iso) version, utilizing the distance comparison function, across both RCNP and FullRCNP models. The table provides metrics for both the interpolation (INT) and out-of-input-distribution (OOID) performance. We show mean and (standard deviation) obtained from 10 runs with different seeds. Statistically significantly best results are bolded.
EQ KL divergence(↓)	Matérn-5/2KL divergence(↓)
dx=2	dx=3	dx=5	dx=2	dx=3	dx=5
INT	RCNP (sta)	0.26 (0.00)	0.40 (0.01)	0.45 (0.00)	0.30 (0.00)	0.39 (0.00)	0.35 (0.00)
RGNP (sta)	0.03 (0.00)	0.05 (0.00)	0.11 (0.00)	0.03 (0.00)	0.05 (0.00)	0.11 (0.00)
RCNP (iso)	0.32 (0.00)	0.41 (0.00)	0.46 (0.00)	0.34 (0.00)	0.39 (0.00)	0.36 (0.00)
RGNP (iso)	0.12 (0.00)	0.17 (0.00)	0.30 (0.00)	0.12 (0.00)	0.17 (0.00)	0.24 (0.00)
FullRCNP (sta)	0.25 (0.00)	0.30 (0.00)	0.36 (0.00)	0.30 (0.00)	0.31 (0.00)	0.29 (0.00)
FullRGNP (sta)	0.03 (0.00)	0.04 (0.00)	0.09 (0.00)	0.03 (0.00)	0.04 (0.00)	0.10 (0.00)
FullRCNP (iso)	0.26 (0.00)	0.31 (0.00)	0.35 (0.00)	0.30 (0.00)	0.32 (0.00)	0.29 (0.00)
FullRGNP (iso)	0.08 (0.00)	0.14 (0.00)	0.25 (0.00)	0.09 (0.00)	0.16 (0.00)	0.21 (0.00)
OOID	RCNP (sta)	0.26 (0.00)	0.40 (0.01)	0.45 (0.00)	0.30 (0.00)	0.39 (0.00)	0.35 (0.00)
RGNP (sta)	0.03 (0.00)	0.05 ( 0.00)	0.11 (0.00)	0.03 (0.00)	0.05 (0.00)	0.11 (0.00)
RCNP (iso)	0.32 (0.00)	0.41 (0.00)	0.46 (0.00)	0.34 (0. 00)	0.39 (0.00)	0.36 (0.00)
RGNP (iso)	0.12 (0.00)	0.17 (0.00)	0.30 (0.00)	0.12 (0.00)	0.17 (0.00)	0.24 (0.0 0)
FullRCNP (sta)	0.25 (0.00)	0.30 (0.00)	0.36 (0.00)	0.30 (0.00)	0.31 (0.00)	0.29 (0.00)
FullRGNP (sta)	0.03 (0.00)	0.09 (0.00)	0.03 (0.00)	0.04 (0.00)	0.10 (0.00)
FullRCNP (iso)	0.26 (0.00)	0.31 (0.00)	0.35 (0.00)	0.30 (0.00)	\( {0.32}\left( {0.00}\right) \)	0.29 (0.00)
FullRGNP (iso)	0.08 (0.00)	0.14 (0.00)	0.25 (0.00)	0.09 (0.00)	0.16 (0.00)	0.21 (0.00)
where NA indicates that no kernel is chosen, i.e., a term in the sum can consist of a single kernel or be missing completely. This setup is based on the kernel learning setup in [35].
See Section D.2 for the precise definition of each kernel. Note that each setting is strictly more general than the one before and entails its predecessor as a special case. Setting  $(iv)$  was reported in the main text.
E.3 Bayesian optimization
After training, the neural process models were evaluated in a Bayesian optimization task. Starting from a common set of five random initial observations,  $\mathbf{x}_0\in \mathcal{U}(0,1)^{\bar{d}_x}$ , each optimization run performed fifty steps of requesting queries by maximizing the expected improvement acquisition function:
$$
\operatorname {E I} (x) = \operatorname {E} \left[ \max (f (x) - f _ {\mathrm {b e s t}}, 0) \right],
$$
where  $f(\cdot)$  is the negative target function (as we wanted to minimize  $f$ ),  $f_{\mathrm{best}}$  is the current observed optimum, and the expectation is calculated based on the neural process or GP model predictions. The acquisition function was optimized via PyCMA [17], the Python implementation of Covariance Matrix Adaptation Evolution Strategy (CMA-ES) [16]. Each CMA-ES run was initialized from the best acquisition function value estimated over a random subset of 100 locations from the hypercube. After each step, the newly queried point and its function evaluation were used to update the surrogate model. For neural process models, this means that the point was added to the context set, whereas for the the GP model, this means that the model was also reinitialized and its hyperparameters refit via type-II maximum likelihood estimation. The neural process model parameters were kept fixed without retraining throughout the Bayesian optimization task.
The results reported in Figure 2 in the main paper were computed using a single pretrained neural process model and ten random Bayesian optimization restarts for each model. For the full results presented here, each neural process model was pretrained three times and used as a surrogate model in ten optimization runs initialized with different sets of five observations, giving us a total of thirty optimization runs with each model. See Figure S1 and Figure S2 for the results and their respective discussion.
E.4 Computation time
Each model was trained on a single GPU, requiring about  $90 - 120\mathrm{min}$  per run. The subsequent CMA-ES optimization and analysis took about  $60 - 90\mathrm{min}$  running on a CPU with eight cores per experiment. The total runtime was approximately  $280\mathrm{h}$  of GPU time and approximately  $7\mathrm{h}$  on the CPUs.
Figure S1: Ablations on Hartmann 3d. GNP struggled throughout all four pretraining scenarios, while its attentive and relational variants (AGNP, RGNP) were able to benefit from the additional flexibility, greatly improving their performance. CNP struggled to learn as the flexibility in the context sets increases, while its attentive and relational counterparts adapted themselves and were able to converge to optimal performance even in the most restrictive pretraining scenario.
F Details of Lotka-Volterra model experiments
We report here details and additional results for the Lotka-Volterra model from Section 5.3 of the main text.
F.1 Models
The RCNP architecture used in this experiment is modeled on the multi-output CNP architecture proposed in previous work [3, Appendix F]. The multi-output CNP encodes the context set associated with each output as a 256-dimensional vector and concatenates the vectors. The encoder network used in this model is an MLP with three hidden layers and 256 hidden units per layer, and the decoder network is an MLP with six hidden layers and 512 hidden units per layer. The RCNP model replicates this architecture by encoding the targets with respect to the context set associated with each output as 256-dimensional vectors, concatenating the vectors, and using encoder and decoder networks that are the same as the encoder and decoder networks used in the CNP model. We note that the encoder
Figure S2: Ablations on Hartmann 6d. As in the 3d case (see Figure S1), CNP and GNP struggled to learn at all. Similarly, the advanced attentive and relational CNP models converged to their optimum even with the most restrictive prior kernel set. As the flexibility increased, RGNP was able to benefit the most and improved both upon AGNP as well as their conditional NP variants (ACNP, RCNP).
network dimensions are the same as used in the synthetic regression experiments (Section D.1) while the decoder network has more hidden units per layer in the multi-output architectures. The encoder and decoder networks use ReLU activation functions.
The models evaluated in these experiments include RCNP, CNP, ACNP, and ConvCNP as well as RGNP, GNP, AGNP, and ConvGNP. RCNP and RGNP models share the architecture described above while the other models are evaluated with the architectures proposed in previous work [3]. However, in contrast to previous work, where CNP, GNP, ACNP, and AGNP models encoded the context associated with each output with a separate encoder, here we used a shared encoder in RCNP, RGNP, CNP, GNP, ACNP, and AGNP. We made this choice based on preliminary experiments carried out with synthetic multi-output data, where we observed that using a shared encoder improved both RCNP and CNP results. We note that all models are extended with a nonlinear transformation to take into account that the population counts modeled in this experiment are positive [3].
The neural process models were trained with simulated data and evaluated with simulated and real data. The models were trained for 200 epochs with  $2^{14}$  datasets in each epoch and learning rate  $10^{-4}$ . The validation sets used in training included  $2^{12}$  datasets also generated based on the simulated data. The evaluation sets generated based on simulated data included  $2^{12}$  datasets each and the evaluation sets generated based on real data  $2^{8}$  dataset each, and both simulated and real data were used to generate three separate evaluation sets to represent the three task types considered in this experiment: interpolation, forecasting, and reconstruction.
F.2 Data
The datasets used in this task included simulated and real datasets. The simulated datasets were constructed based on simulated predator and prey time series generated using stochastic Lotka-Volterra equations. The Lotka-Volterra equations introduced independently by Lotka [28] and Volterra [42] model the joint evolution between prey and predator populations as
$$
\left\{ \begin{array}{l} \dot {u} = \alpha u - \beta u v \\ \dot {v} = \delta u v - \gamma v \end{array} \right.,
$$
where  $u$  denotes the prey,  $v$  denotes the predators,  $\alpha$  is the birth rate of the prey,  $\beta$  is the rate at which predators kill prey,  $\delta$  the rate at which predators reproduce when killing prey, and  $\gamma$  is the death rate of the predators; the dot denotes a temporal derivative, i.e.  $\dot{u} \equiv \frac{\mathrm{d}}{\mathrm{d}t} u$ . In this work, we generated simulated data with a stochastic version proposed by [3],
$$
\left\{ \begin{array}{l} \mathrm {d} u = \alpha u \mathrm {d} t - \beta u v \mathrm {d} t + u ^ {1 / 6} \sigma \mathrm {d} W ^ {u} \\ \mathrm {d} v = \delta u v \mathrm {d} t - \gamma v \mathrm {d} t + v ^ {1 / 6} \sigma \mathrm {d} W ^ {v} \end{array} \right.,
$$
where  $W^{u}$  and  $W^{v}$  are two independent Brownian motions that add noise to the population trajectories and  $\sigma > 0$  controls the noise magnitude.
We simulated trajectories from the above equation using the code released with the previous work [3]. The parameters used in each simulation were sampled from:  $\alpha \sim \mathcal{U}(0.2, 0.8)$ ,  $\beta \sim \mathcal{U}(0.04, 0.08)$ ,  $\delta \sim \mathcal{U}(0.8, 1.2)$ ,  $\gamma \sim \mathcal{U}(0.04, 0.08)$ , and  $\sigma \sim \mathcal{U}(0.5, 10)$ . The initial population sizes were sampled from  $u, v \sim \mathcal{U}(5, 100)$ , and the simulated trajectories were also scaled by a factor  $r \sim \mathcal{U}(1, 5)$ . To construct a dataset, 150-250 observations were sampled from the simulated prey time series and 150-250 observations from the simulated predator series. The selected observations were then divided into context and target observations based on the selected task type which was fixed in the evaluation sets and selected at random in the training and validation data.
The datasets representing interpolation, forecasting, or reconstruction tasks were created based on the selected simulated observations as follows. To create an interpolation task, 100 predator and 100 prey observations, selected at random, were included in the target set, and the rest were included in the context set. To create a forecasting task, a time point was selected at random between 25 and 75 years, and all observations prior to the selected time point were included in the context set and the rest were included in the target set. To create a reconstruction task, a random choice was made between prey and predators such that the observations from the selected population were all included in the context set while the observations from the other population were divided between the context and target sets by choosing at random a time point between 25 and 75 years and including all observations prior to the selected time point in the context set and the rest in the target set.
The real datasets used to evaluate the neural process models in the sim-to-real tasks were generated based on the famous hare-lynx data [32]. The hare and lynx time series include 100 observations at regular intervals. The observations were divided into context and target sets based on selected task types, and the same observations were used to generate separate evaluation sets for each task. The forecasting and reconstruction task was created with the same approach that was used with simulated data, while the interpolation task was created by choosing at random 1–20 predator observations and 1–20 prey observations that were included in the target set and including the rest in the context set.
F.3 Full results
We evaluate and compare the neural process models based on normalized log-likelihood scores calculated based on each evaluation set and across training runs. We compare RCNP, CNP, ACNP, and ConvCNP models in Table S9 (a) and RGNP, GNP, AGNP, and ConvGNP models with 32 basis functions in Table S9 (b) and with 64 basis functions in Table S9 (c). We observe that the best model depends on the task, and comparison across the tasks indicates that RCNP or RGNP are not consistently better or worse than their convolutional and attentive counterparts. This is not the case with CNP and GNP, and we observe that the other models including RCNP and RGNP are almost always substantially better than CNP or GNP. The models with 64 basis functions performed on average better than the models with 32 basis functions, but we observed that some RGNP, GNP, and AGNP training runs did not converge and most GNP runs did not complete 200 epochs due to numerical errors when 64 basis functions were used. We therefore reported results with 32 basis functions in the main text.
Table S9: Normalized log-likelihood scores in the Lotka-Volterra experiments (higher is better). We compared models that predict (a) mean and variance and models that also predict a low-rank covariance matrix with (b) 32 basis functions or (c) 64 basis functions. The mean and (standard deviation) reported for each model are calculated based on 10 training outcomes evaluated with the same simulated (S) and real (R) learning tasks. The tasks include interpolation (INT), forecasting (FOR), and reconstruction (REC). Statistically significantly best results are bolded. RCNP and RGNP models perform on par with their convolutional and attentive counterparts, and nearly always substantially better than the standard CNPs and GNPs, respectively.
INT (S)	FOR (S)	REC (S)	INT (R)	FOR (R)	REC (R)
(a)	RCNP	-3.57 (0.02)	-4.85 (0.00)	-4.20 (0.01)	-4.24 (0.02)	-4.83 (0.03)
ConvCNP	-3.47 (0.01)	-4.85 (0.00)	-4.06 (0.00)	-4.21 (0.04)	-5.01 (0.02)
ACNP	-4.04 (0.06)	-4.87 (0.01)	-4.36 (0.03)	-4.18 (0.05)	-4.79 (0.03)
CNP	-4.78 (0.00)	-4.88 (0.00)	-4.86 (0.00)	-4.74 (0.01)	-4.81 (0.02)
(b)	RGNP	-3.51 (0.01)	-4.27 (0.00)	-3.76 (0.00)	-4.31 (0.06)	-4.47 (0.03)
ConvGNP	-3.46 (0.00)	-4.30 (0.00)	-3.67 (0.01)	-4.19 (0.02)	-4.61 (0.03)
AGNP	-4.12 (0.17)	-4.35 (0.09)	-4.05 (0.19)	-4.33 (0.15)	-4.48 (0.06)
GNP	-4.62 (0.03)	-4.38 (0.04)	-4.36 (0.07)	-4.79 (0.03)	-4.72 (0.04)
(c)	RGNP	-3.54 (0.06)	-4.15 (0.10)	-3.75 (0.10)	-4.30 (0.04)	-4.44 (0.02)
ConvGNP	-3.46 (0.00)	-4.15 (0.01)	-3.66 (0.01)	-4.20 (0.03)	-4.57 (0.05)
AGNP	-4.06 (0.10)	-4.16 (0.04)	-3.95 (0.11)	-4.34 (0.10)	-4.41 (0.06)
GNP	-4.63 (0.03)	-4.36 (0.05)	-4.40 (0.05)	-4.82 (0.01)	-4.75 (0.04)
F.4 Computation time
When the models were trained on a single GPU, each training run with RCNP and RGNP took around 6.5–7 hours and each training run with the reference models around 2–3 hours depending on the model. The results presented in this supplement and the main text required 120 training runs, and 10–15 training runs had been carried out earlier to confirm as a sanity check our ability to reproduce the results presented in previous work. The multi-output models used in this experiment were additionally studied in preliminary experiments using synthetic regression data. The computation time used in all the experiments is included in the estimated total reported in Section C.2.
G Details of Reaction-Diffusion model experiments
We report here details and additional results for the Reaction-Diffusion model from Section 5.4 of the main text.
G.1 Models
We compared RCNP, RGNP, ACNP, AGNP, CNP, and GNP models trained using simulated data with input dimensions  $d_{x} = \{3,4\}$ . We mostly used the architectures described in Section D.1, but since less training data was used in this experiment, we reduced the number of hidden layers in the decoder network to 4. The models were trained for 100 epochs with  $2^{10}$  datasets in each epoch and learning rate  $10^{-4}$ . The validation sets used in training included  $2^{8}$  datasets and the evaluation sets used to compare the models in completion and forecasting tasks included  $2^{8}$  datasets each.
G.2 Data
We generated simulated data using a simplified equation inspired by the cancer evolution equation from Gatenby and Gawlinski [13]. The equation involves three quantities: the healthy cells  $(\mathbf{z},t)\mapsto u(\mathbf{z},t)$ , the cancerous cells  $(\mathbf{z},t)\mapsto v(\mathbf{z},t)$ , and an acidity measure  $(\mathbf{z},t)\mapsto w(\mathbf{z},t)$ , with
$t \in [0, T]$  and  $\mathbf{z} \in E \subset \mathbb{R}^2$ . Their temporal and spatial dynamics are described by:
$$
\left\{ \begin{array}{l} \dot {u} = r _ {u} u \left(1 - u / k _ {u}\right) - d _ {u} w \\ \dot {v} = r _ {v} v \left(1 - v / k _ {v}\right) + D _ {v} \nabla^ {2} v \\ \dot {w} = r _ {w} v - d _ {w} w + D _ {w} \nabla^ {2} v \end{array} , \right. \tag {S8}
$$
where  $r_u$ ,  $r_v$ , and  $r_w$  are apparition rates,  $k_u$  and  $k_v$  control the maximum for the number of healthy and cancerous cells,  $d_u$  and  $d_w$  are death rates relative to other species, and  $D_v$  and  $D_w$  are dispersion rates.