$\infty$-Diff: Infinite Resolution Diffusion with Subsampled Mollified States
Authors: Sam Bond-Taylor
Chris G. Willcocks
Venue: ICLR 2024 poster (2024)
Keywords: Generative Models
Diffusion Models
Infinite Resolution
Primary area: generative models

Abstract: This paper introduces $\infty$-Diff, a generative diffusion model defined in an infinite-dimensional Hilbert space, which can model infinite resolution data. By training on randomly sampled subsets of coordinates and denoising content only at those locations, we learn a continuous function for arbitrary resolution sampling. Unlike prior neural field-based infinite-dimensional models, which use point-wise functions requiring latent compression, our method employs non-local integral operators to map between Hilbert spaces, allowing spatial context aggregation. This is achieved with an efficient multi-scale function-space architecture that operates directly on raw sparse coordinates, coupled with a mollified diffusion process that smooths out irregularities. Through experiments on high-resolution datasets, we found that even at an $8\times$ subsampling rate, our model retains high-quality diffusion. This leads to significant run-time and memory savings, delivers samples with lower FID scores, and scales beyond the training resolution while retaining detail.

OO-DIFF INFINITE RESOLUTION DIFFUSION WITH SUBSAMPLED MOLLIFIED STATES
Sam Sond-Taylor, Chris G. Willcocks
Department of Computer Science
Durham University (samuel e bond-taylor, christopher g willcocks @durham. ac uk
ABSTRACT
This paper introduces oc-Diff, a generative diffusion model defined in an infinite- dimensional Hilbert space, which can model infinite resolution data. By training on randomly sampled subsets of coordinates and denoising content only at those loca- tions, we learn a continuous function for arbitrary resolution sampling. Unlike prior neural field-based infinie-diimensinaa models, which use point-wise functions requiring latent compression, our method employs non-local integral operators map between Hilbert spaces, allowing spatial context aggregation. This is achieved with an efficient multi-scale function-space architecture that operates directly on sparse coordinates, coupled with mollified diffusion process that smooths out irregularities. Through experiments on high-resolution datasets, we found that even an 8x subsampling rate, our model retains high-quality diffusion. This leads to significant run-time and memory savings, delivers samples with lower FID scores, and scales beyond the training resolution while retaining detail.
INTRODUCTION
Denoising diffusion models (Song and Ermon, 2019; Ho et al., 2020) have become a dominant choice for data generation, offering stable training and the ability to generate diverse and high quality samples. These methods function by defining forward diffusion process which gradually destroys information by adding Gaussian noise, with neural network then trained to denoise the data, in turn approximating data distribution. Scaling diffusion models to higher resolutions has been the topic of various recent research, with approaches including iteratively upsampling lower resolution images (Ho et al., 2022a) and operating in a compressed latent space (Rombach et al., 2022).
Deep neural networks typically assume that data can be represented with fixed uniform grid, however, the underlying signal is often continuous. As such, these approaches scale poorly with resolution. Neural fields (Xie et al., 2022; Sitzmann al., 2020; Mildenhall et al., 2021) address this problem by directly representing data as mapping from coordinates to intensities (such as pixel values), making the parameterisation and memory/run-tin allowing training on data that would not usually fit in memory. Neural field-based generative models (Dupont al., 2022a;b; Bond- Taylor and Willcocks, 2021; Du al., 2021) have been developed to take advantage of these properties. Being inherently independent between coordinates, these
from resolution independent from resolution, thereby
Neural Operators
Sample at any resolution (trained at resolution of red box)
(c)
p( (zx))
(o(C)
Figure I: We define a diffuuson process in an infinite dimensional image space by randomly sampling coordinates and training a model parameterised by neural operators denoise at those coordinates.
Source code is available https::/ http://ggtttuuutto http:/ccc//mmmm/tteeeeee www.mabbtttttttttttteeeaa

---

Figure 2: Modelling data functions allows sampling at arbitrary resolutions using the same model with different sized noise. Left to right: 64x64, 128x128 256x256 (original), 512x512, 1024x1024.
approaches condition networks on compressed latent vectors to provide global information. However, the sample quality of these methods is significantly lower than miiteedimensional generative models.
In this work we develop an approach that substantially improves upon the quality and scaling of existing infinite dimensional generative models, reducing the gap while retaining the benefits of infinite dimensional models: subsampling coordinates to decouple memory/run-time from resolution, making scaling training and sampling at arbitrary resolutions. We achieve this by designing a Gaussian diffusion model in an infinite dimensional state space'. We argue that compressed latent-based neural fields cannot effectively be used to parameterise such diffusion models due to the reliance on compression, going against standard diffusion architecture design, with it also being impractical compress states to latents at every step. Instead, we propose using non-local integral operators to model the denoising function, aggregating both global and local information in order to effectively denoise the data.
dimensionn methods,
computationally feasible, while also allowing
Specifically, we propose oo-Diff, addressing these issues:
We introduce Gaussian diffusion model defined in an infinite infinite-dimensionn dimensional state space that allows complex arbitrary resolution data to be generated (see Fig. 2)
We design powerful and scalable, function-space architecture that operates directly raw sparsely subsampled coordinates, enabling improvements in run-time and memory usage.
We achieve state-of-the-are FID scores multiple high-res image datasets, trained with up 8x subsampling, substantially outperforming prior infinite resolution generative models.
BACKGROUND
Here we review Gaussian diffusion models (Section 2.1) and generative neural fields (Section 2.2).
2.1 DIFFUSION MODELS
Gaussian diffusion models (Sohl-Dickstee et al., 2015; Ho al., 2020) are formed by defining a forward process q(<1,r(<0)) that gradually adds noise to the data, ænn ~ g(xo), over T steps, resulting in sequence of latent variables æ1,.... 2r such that g(ær) & (xT: 0, I). The reverse of this process also be expressed Markov chain p( ((Fo.)). ). Choosing Gaussian transition densities
chosen to ensure these properties hold, the densities may be expressed as
T
q( (mp) = II
((x1))))1)) t=1
((xa/ar-1) = -N(x; v B,00-11
T
p(Zo.T) = p((T) II p(z,-1 12,,) Am1
p(z,-1/Z,) =
En Ez t)),
(2)
where 0 < BI,...,, By < 1 is pre-defined variance schedule and the covariance is typically of the form Eo(2,, 1) of1 where often of B. Aiding training efficiency, (20) can be expressed in closed form as ((x:(aap)
N(n;; Varao, (1 T) where a, = I a, for as Training is possible by optimising the evidence lower bound on the negative log- ligliike,,oodd, expressed as the KL-divergence between the forward process posteriors and backward transitions at each step
Eng (Dxa(q(2)111 mo) (p(p)pp(ppp))))))) (x:))
Σ Eg
(1, mn) - (=(
[20
A number of parallel works including Kerrigan et al. (2023); Lim et al. (2023); and Franzese al. (2023) also developed infinite-iimensional diffusion models that non-local integral operators, see Sec. 6 for more.
2

---

for q(21-1|21, (2 fá, (xt, Ty), 32 where μu, and Be can be derived in closed form. is typical to reparameterise Ho to simplify the variational bound, one example of such found to improve visual quality is (, t) = a. (a: vi-a, co(2;, t)), where the denoising network instead predicts the added noise. Diffusion models are closely connected with score-matching models; this can be seen by making the approximation (De Bortoli al., 2021).
p (z;) p(z)// exp(
logp(zz))
(4)
2 (x:11: VI- B + 6. Ve, log p(ge B,J),
(5)
which holds for small values of 3,. While va, p(r,) is not available, it can be approximated using denoising score matching methods (Hyvärinen, 2005; Vincent, 2011). Given that Va, log p(;;) Ep( Va. log p( (2, Øo)] we can learn an approximation to the score with a neural network parameterised by 8, so Dz t) = V log p(x;) (Song and Ermon, 2019), by minimising a reweighted variant of the ELBO (Eq. 3). One problem with diffusion models is the slow sequential sampling; to speed this up, denoising diffusion implicit models (DDIMs) transform diffusion models into deterministic models allowing fewer steps yield the same quality, replacing the sampling steps with
04E6 (1 1)
-t V
V 04-1 <00
04
2.2 GENERATIVE NEURAL FIELDS
Neural fields (Xie et al., 2022; Mildenhall et al., 2021; Sitzmann al., 2020) are approach for continuous data representation that map from coordinates c to values v (such as RGB intensities), fa(c) This decouples the memory needed to represent data from its resolution. The mapping function, fe. typically an MLP network, is optimised by minimising a reconstruction loss with ground truth values at each coordinate. The local nature of So allows the loss to be Monte-Carlo approximated by evaluating fe on subsets of coordinates, allowing higher resolution data than would fit memory be trained on. Since fo is independent per coordinate, being unable transform over multiple points like nonvoluiions/ransfoomni generally also condition on compressed latent vectors used to describe single data points, fo z). Dupont et al. (2022a) first uses meta meta-learning to compress the dataset into latent conditional neural fields, then approximates the distribution of latents with DDPM (Ho et al., 2020) or Normalizing Flow (Rezende and Mohamed, 2015). Bond-Taylor and Willcocks (2021) form VAE-like model with a single gradient step to obtain latents. Zhuang et al. (2023) design a diffusion model with a small subset of coordinates used to provide context. Finally, some approaches use hypemnetwrrss to output the weight of neural fields: Dupont et al. (2022b) define the hypernetwork as a generator in an adversarial framework, and Du al. (2021) use manifold learning to represent the latent space.
to represent spaces of functions approaches
INFINITE DIMENSIONAL DIFFUSION MODELS
In this section we extend diffusion models to infinite dimensions in order to allow higher-resolution data to be trained on by subsampling coordinates during training and permit training/sampling at arbitrary resolutions. We argue that application of conditional neural fields to diffusion models is problematic due to the need to compress to a latent vector, adding complexity and opportunity for error, instead, the denoising function should be non-local integral operator with no compression. A number of parallel works also developed diffusion models in infinite dimensions, including Kerrigan al. (2023); Lim et al. (2023); and Franzese et al. (2023); we recommend also reading these works, which go further in the theoretical treatment, while ours focuses more on design and practical scaling.
To achieve this, we restrict the diffusion state space to a Hilbert space H. elements of which, E H. are functions, e.g. x: R. - Rd. Hilbert spaces are equipped with an inner product (---) and corresponding norm ||- 1|x. For simplicity we consider the case where 7. the space of L functions from [0. 1]"d to Rd although the following sections can be applied to other spaces. As such, a point in H could represent an image, audio signal, video, 3D model, etc. Gaussian measure μ can be defined in H in terms of its characteristic function á (Da Prato and Zabczyk, 2014)),,,0114),
(7) where the mean m lies in H. m E H and the covariance operator (C : H. + H) is self-adjoint (denoted C = C*), non-negative (i.e. C > 0), and trace-class (ʃx [==())))(())) tr(C) < oo)
(xx) = exp (i(x, m) + 2 (Cx, x)),
3

---

White Noise
Diffusion
Mollified
Diffusion
Example diffusion processes. Mollified diffusion smooths diffusion states allowing the space to be more effectively modelled with continuous operators.
Figure 3:
(Kukush, 2020). For Gaussian random element with distribution μ., z ~ N(m, C). The Radon- Nikodym theorem states the existence of a density for measure absolutely continuous with respect to base measure j:: for example, the density between two Gaussians is given by Minh (2021); )); see Lim et al. (2023); Kerrigan et al. (2023) for more detail in the context of functional diffusion models.
3.1 MOLLIFICATION
When defining diffusion in infinite dimensions, it may seem natural to use white noise in the forwards process, where each coordinate is an independent and identically distributed Gaussian random variable; that is, (V(0C)) where Cj (2((s), 2ss)) = 5(s s'), using the Dirac delta function 5. However, this noise does lie in H (Da Prato and Zabezyk, 2014) with not satisfying the trace- class requirement. Instead, obtain Gaussian noise in H by convolving white noise with mollifier kernel ks()) > 0 corresponding to a linear operator T, giving (0, T7*),, smoothing the white noise to lie in H (Higdon, 2002). To ensure one-to-one correspondence between kernel and noise, k must satisfy SRa k(s)ds < oo and Jea (s)ds < co, making TT* self-adjoint and non-negative. Considering i to be Gaussian kernel with smoothing parameter > 0, h Tx is given by
f K(c y. ()y() dy, where K(y, (4+1) 2
h(c)=
(8)
3.2 INFINITE DIMENSIONAL MOLLIFIED DIFFUSION
To formulate a diffusion model in H. we must specify the transition distributions. However, irregu- larity in data points can impact stability, leading to the model being unable to generalise across different subsampling ratcs/resolutions. This can be mitigated by careful hyperparameter tuning or, in our case, by also mollifying x (as with the previous noise mollification). While the necessity of this depends on the nature of x, we have included for completeness. First, we define the marginals
g(<x (ro) = N(x; VorToo (1 - ","",",
where coefficients are the same as in Section 2.1. From this we are able to derive a closed form representation of the posterior (proof in Appendix B.1),
g(F1-11F2, 20) Nx (x vo), (o) B,TT),
where pe( 20) Va Two var(1 - Ōt-1)
1 - ā
(10)
and B,
8r.
1-ar
Defining the reverse transitions as po (2 (xx)) = -N (2 1), TT,*, then can parame- terise μ4: * + H to directly predict wo. The loss in Eq. (3) can be extended to infinite dimensions
(Pinski et al., 2015). However, since Ho et al. (2020) find that predicting the noise yields higher image quality, we parameterise μe to predict & ~N(00,TT) motivated by the rewriting the loss as
L- Eyg
(x
VI ai :)
B.
140 (2te 1) ) 7.
(11)
207
Br
Mo
Mo
Dt
fo(
(12)
/ 0
VT a2
where xz Ie( = /0 + -I- Since T- does not affect the minima, we follow Ho et al. (2020) and use simplified loss, LoSmplo
E.lll.o(ors,)) () = The concurrent work by Kerrigan
4

---

et al. (2023) showed that in the dimeiie-admeniioalll limit, the loss will be finite only for specific choices of 3,, while Lim et al. (2023) found similar only for specific parameterisations of uu: however, since the loss is Monte-Carlo approximated, this is not problematic in practice.
Data Mollification By mollifying the training data 2o to ensure regularity, resulting samples similarly regular; directly predicting Tig would give estimate of the original data, but by predicting & we are only able to sample Tap- However, in the case of the Gaussian mollifier kernel with adequate boundary conditions, the existence of the inverse T is clear if we consider the Fourier transform of x(c), denoted i(w), then the Gaussian convolution can be defined by h(w) = =(()) And so TI is one-to-one any class of Fourier transformable functions, with Tr being bounded ensuring uniqueness and therefore invertibility (John, 1955). Explicitly, the inverse is given by f(w)= = - h(w) (Hummel et al., 1987). However, inverting is ill-conditioned, with arbitrarily small changes (for instance by floating point error) destroying smoothness (Hummel et al., 1987). In this case, the Wiener filter can for instance be used as an approximate inverse, defined as e-u²t
i(W) = 2ue M(w), where is an estimate of the inverse SNR (Biemond et al., 1990).
4 PARAMETERISING THE DIFFUSION PROCESS
In order to model the denoising function in Hilbert space, there are certain properties that is essential for the class of learnable functions to satisfy so as allow training on infinite resolution data:
1. Can take as input points positioned at arbitrary coordinates.
2. Generalises to different numbers of input points than trained on, sampled on regular grid.
3. Able to capture both global and local information.
4. Scales to very large numbers of input points, i.c. efficient in terms of runtime and memory.
Recent diffusion models often a U-Net (Ronneherger et al., 2015) consisting of convolutional encoder and decoder with skip-connections between resolutions allowing both global and local information to be efficiently captured. Unfortunately, U-Nets function on fixed grid making them unsuitable. However, we can take inspiration to build an architecture satisfying the desired properties.
4.1 NEURAL OPERATORS
Neural Operators (Li et al.. 2020; Kovachki et al., 2021) are a framework designed for efficiently solving partial differential equations by learning to directly map the PDE parameters to the solution in a single step. However, more generally they are able to learn map between two infinite dimensional function spaces making them suitable for parameterising an infinite dimensional diffusion model.
Let X' and be separable Banach spaces representing the spaces of noisy and denoised data re- spectively; neural operator is map Fo: X' + S. Since x E X' and 8 E S are both functions, we only have access to pointwise evaluations. Let e E (D) be an m-point discretisation of the
domain D= [0, 1j" (i.e. is m coordinates), and assume we have observations x(c) E mxd To be discretisation invariant, the neural operator may be evaluated at any e E D, potentially e & c, thereby allowing a transfer of solutions between discretisations. Each layer is built using non-local integral kernel operator, K(x;; o), parameterised by neural network Kgg aggregating information spatially,
(K(a;; = J, Kolc, b, (())))))
Vc E D.
(13)
Deep networks can be built in a similar manner to conventional methods, by stacking layers of linear operators with non-linee -onear activation functions, to + 01 .... UL where i + 0 is defined as
(c) = oW (c) + (KCa:; (c)),
Vee E D,
(14)
for input 0gg = activations f/ output ey = 8, pointwise linear transformation W: Rdd - TR. and activation function a: -R. One example the Fourier Neural Operator (FNO) (Li et al., 2021).
(K(x; (c) g- (Rs (Gv.)) (c),
Vc E D,
(15)
where g is the Fourier transform, and R. is learned transformation in Fourier space. When coordinates lie on regular grid, the fast Fourier transform can be used, making FNOs fast and scalable.
5

---

Sparse Neural Operators
x(c-
y)u(y)dy, Vee D
NC)
W
k-NN
k-NN lerp
lerp
Figure 4: oo-Diff uses a hierarchical architecture that operates irregularly sampled functions at the top level to efficiently capture fine details, and on fixed grids at the other levels to capture global structure. This approach allows scaling to intricate high-resolution data.
4.2 MULTI-SCALE ARCHITECTURE
While neural operators which satisfy all required properties (1-4) exist, such Galerkin attention (Cao, 2021) (softmax-fi free linear attention) and MLP-Mixers (Tolstikhin et al., 2021), scaling beyond small numbers of coordinates is still challenging due to the high memory costs. Instead we design U-Net inspired multi-scale architecture (Fig. 4) that separately aggregates local/global information.
In continuous setting, there two main approaches to downsampling: (1) selecting subset of coordinates (Wang and Golland, 2022) and (2) interpolating points to a regularly spaced grid (Rahman al., 2022). We found that with repeated application of (1), approximating integral operators non-uniformly spaced grids with very few points did not perform nor generalise well, likely due to the high variance. On the other hand, while working with regular grid removes some sparsity properties, issues with variance are much lower. As such, we use a hybrid approach with sparse operators applied on the raw irregularly sampled data local regions; after this points are interpolated a regular grid and grid-based architecture is applied in order to aggregate global information; if the regular grid is of sufficiently high dimension, this combination should be sufficient. While an FNO (Li al., 2021; Rahman et al., 2022) architecture could be used, we achieved better results with dense convolutions (Nichol and Dhariwal, 2021), with sparse operators used for resolution changes.
4.3 EFFICIENT SPARSE OPERATORS
At the sparse level convolution operators (Kovachki al., 2021), finding them be more performant than Galerkin attention, with global context provided by the multiscale architecture. This defined using translation invariant kernel restricted to the local region of each coordinate, (c)..
S x(c- ())( dy,
x <cc)
Vc D.
(16)
We restrict 6 to be depthwise kernel due to the greater parameter efficiency for large kernels (particularly for continuously parameterised kernels) and finding that they more able generalise when trained with fewer sampled coordinates; although the sparsity ratio is the same for regular and depthwise convolutions, because there are substantially more values in a regular kernel, there is more spatial correlation between values. When very large number of sampled coordinates are used, fully continuous convolutions are extremely impractical in terms of memory usage and run-time. In practice, however, images are obtained and stored on a discrete grid. As such, by treating images as high dimensional, but discrete entities, we can take advantage efficient sparse convolution libraries (Choy et al., 2019; Contributors, 2022), making memory usage and run-times much more reasonable. Specifically, we use TorchSparse (Tang et al., 2022), modified to allow depthwise convolutions. Wang and Golland (2022) proposed using low discrepancy coordinate sequences approximate the integrals due to their better convergence rates. However, we found uniformly sampled points to be more effective, likely because high frequency details are able to be more easily captured.

---

pe gpick
Figure 5: Samples from oc-Diff models trained on sets of randomly subsampled coordinates.
Method
CelebAHQ-64
CelebAHQ-122 128
FFHQ-256
Church-256
Dimen-DDieeiionnll
CIPS (Anokhin et al., 2021)
5.29
10.80
StyleSwin (Zhang et al., 2022)
3.39
3.25
8.28
UT (Bond-Taylor et al., 2022)
3.05
5,52
StyleGAN2 (Karras et al., 2020)
2.20
2.35
6.21
D2F (Dupont et al., 2022a)
40.4*
DPF (Zhuang et al., 2023)
13.21
GEM (Du et al., 2021)
14.65
23.73
35.62
87.57
GASP (Dupont et al., 2022b)
9.29
27.31
24.37
37.46
oc-Diff (Ours)
4.57
3.02
3.87
10.36
Table FIDCLIP (Kynkäänniemi al., 2023) evaluation against finite-dimensional methods well as other infiniteddimensional approaches which are trained on coordinate subsets. "=Inception FID.
EXPERIMENTS
In this section demonstrate that the proposed mollified diffusion process modelled with neural operator based networks and trained on coordinate subsets are able to generate high quality, high resolution samples. We explore the properties of this approach including discretisation invariance, the impact of the number of coordinates during training, and compare the sample quality of our approach with other infinite dimensional generative models. We train models on 256 256 datasets, FFHQ (Karras et al., 2019) and LSUN Church (Yu et al., 2015), well as CelebA-HQ (Karras et al., 2018); unless otherwise specified models are trained on 1/4 of pixels (to fit in memory), randomly selected.
Very large batch sizes are necessary to train diffusion models due the high variance (Hoogeboom al., 2023), making training on high resolution data on single GPU impractical. To address this, we use diffusion autoencoders (Preechakul et al., 2022) to reduce stochasticity: during training, sparsely sampled data are encoded to small latent vectors and used to condition our pixel-level infinite-dimensionall diffusion model, allowing better estimation of the denoised data at high time steps. Subsequently, a small diffusion model is quickly trained to model these latents. Our encoder is the downsampling part of our architecture (left half of Fig. 4). When sampling. use the deterministic DDIM interpretation with 100 steps. Additional details are in Appendix A. Source code available https: / thub. com/ samb- / /eitt/ifftttt infiin-ffiiiii
Sample Quality Samples from our approach can be found in Fig. which are high quality, diverse, and capture fine details. In Table we quantitatively compare with other approaches that treat inputs as infinite dimensional data, as well as more traditional approaches that assume data lies on fixed grid. As proposed by Kynkäänniemi et al. (2023), we calculate FID (Heusel et al., 2017) using CLIP features (Radford et al., 2021) which is better correlated with human perception of image quality. Our approach scales to high resolutions much more effectively than the other function-bars approaches evidenced by the substantially lower scores. Visual comparison between samples from our approach and other function saaeed approaches can be found in Fig. 6 where samples from our approach be seen be higher quality and display more details without blurring or adversarial
7

---

GASP (Dupont et al., 2022b)
GEM (Du et al., 2021)
Do-Diff (Ours)
DPF (Zhuang et al., 2023)
D2F (Dupont et al., 2022a)
Figure 6: Qualitative comparison with other infinite dimensional approaches.
Architecture
FIDCLIP
Rate
FID
Speedup
FIDeue
512
Sparse Downsample
85,99
1x
3.15
1.0x
10
Nonlinear Kernel
24.49
2x
4.12
1.0x
6
Quasi Monte Carlo
7.63
4x
4.75
1.3x
Regular Convs
5.63
8x
6.48
1.6x
20
40
00
80
100
oo-Diff (Ours)
4.75
Sampling Steps
Table 3: Impact of subsampling rate quality for FFHQ 128. FID calcu- lated with 10k samples.
Figure FID at var-
Table Architectural component ious steps & resolutions. ablations in terms of FIDCLIP-
artefacts. All these approaches are based on neural fields (Xie et al., 2022) where coordinates are treated independently; in contrast, approach uses neural operators to transform functions using spatial context thereby allowing more details to captured. Both GASP (Dupont et al., 2022b) and GEM (Du et al., 2021) rely compressed cirent-coniiiionll hypernetworks which makes efficiently scaling difficult. D2F (Dupont et al., 2022a) relies on a deterministic compression stage which loses detail due the finite vector size. DPF (Zhuang al., 2023) uses small fixed sized coordinate subsets as global context with other coordinates modelled implicitly, thereby causing blur.
Discretisation Invariance In Fig. 2 we demonstrate the discretisation invariance properties of our approach. After training on random coordinate subsets from 256 x 256 images, we can sample from this model at arbitrary resolutions which we show at resolutions from 64 x 64 to 1024 x 1024 by initialising the diffusion with different sized noise. We experimented with (alias-free) continuously parameterised kernels (Romero et al., 2022) but found bi- bi-linearly interpolating kernels to be more effective. At each resolution, even exceeding the training data, samples consistent and diverse. In Fig. 7 we analyse how the number of sampling steps affects quality at different sampling resolutions.
Coordinate Sparsity One factor influencing the quality of samples the number of coordinates sampled during training; fewer coordinates means fewer points from which to approximate each integral. We analyse the impact of this in Table 3, finding that as expected, performance decreases with fewer coordinates, however, this effect is fairly minimal. With fewer coordinates also comes substantial speedup and memory savings; at 256 x 256 with 4x subsampling the speedup is 1.4x.
Architecture Analysis In Table we ablate the impact of various architecture choices against the architecture described in Section 4.2, matching the architecture as closely as possible. particular, sparse downsampling (performed by randomly subsampling coordinates; equidistant subsampling, Qi et al., 2017) fails to capture the distribution. Similarly using a spatially nonlinear kernel (Eq. 13), implemented as conv, activation, conv, does generalise well unlike linear kernels (we observed similar for softmax transformers, Kovachki et al., 2021).
observed similar with
Super-resolution The discretisation invariance properties of the proposed approach makes superresolution a natural application. We evaluate this in a simple way, passing low resolution image through the encoder, then sampling at higher resolution; see Fig. where downside of this specific approach is that information is lost in the encoding process, however, this could potentially by improved by incorporating DDIM encodings (Song al., 2021).
clear that more details have been added. A
Figure 8: Super-resolution
8

---

Inpainting Inpainting is possible with mollified diffusion (Fig. 9), using reconstruction guidance (Ho et al., 2022b), 2;-1 + Ft-1 AV., se |l @ (in(2,,, t) T=)l for impainting mask m, learned estimate of Txn. Po. and image to be inpainted J. The dif- fusion autoencoder framework gives additional level of control when impainting since the reverse diffusion process can be applied to encodings from a chosen time step iss allowing control over how different the inpainted region is from the original image.
Original
t = 200 t, = 350
Figure 9: Inpainting.
6 DISCUSSION
There are a number of interesting directions to improve our approach including more power- ful/ffiiciett neural operators, more efficient sparse methods, better integral approximations, and improved UNet design (Williams et al., 2023). Having demonstrated that diffusion models can be trained with 8x subsampling rates, we believe there is substantial room for further performance gains. Also of interest are recent works which speed up diffusion sampling by iteratively upsampling throughout the backwards process, requiring separate model for each resolution (Jing al., 2022; Zhang et al., 2023); the resolution invariance of our approach permits this with a single model.
Recent diffusion advances are also complementary our approach, these include consistency models (Song et al., 2023), stochastic interpolants (Albergo et al., 2023), Schrödinger bridges (De Bortoli al., 2021), critically-damper diffusion (Dockhorn et al., 2022), architecture improvements (Hooge- boom et al., 2023), and faster solvers (Lu et al., 2022). Similar to our mollified diffusion, blurring has been used to improve diffusion (Rissanen et al., 2023; Hoogeboom and Salimans, 2023). Similar to GASP (Dupont et al., 2022b), other neural field GAN approaches exist such as CIPS (Anokhin et al., 2021) and Poly-INR (Singh et al., 2023), however, these approaches use convolutional discriminators requiring all coordinates on fixed grid, preventing scaling to infinite resolutions. Also of relevance are Neural Processes (Garnelo et al., 2018; Dutordoir et al., 2022) which learn distributions over functions similar to Gaussian Processes, however, these approaches address conditional inference, whereas we construct an unconditional generative model for substantially more complex data.
Concurrent with this work, other papers independently proposed diffusion models in infinite dimen- sions (Lim et al., 2023; Franzese et al., 2023; Hagemann et al., 2023; Zhuang et al., 2023; Kerrigan al., 2023; Baldassari al., 2023; Pidstrigach et al., 2023), these approaches are complementary to ours and distinct in a number of ways. While our work focuses on the practical development necessary to efficiently model complex high-dimensional data, these papers instead focus more on theoretical foundations, typically being only applied to simple data (e.g. Gaussian mixtures and MNIST). Of particular interest, Kerrigan et al. (2023) also develop diffusion models in Hilbert space, going further than our work in foundational theory, including more on the requirements to obtain well-posed models, as well as considering different function spaces; (Lim et al., 2023) develop infinite-dimensiona diffusion defined as SDEs; and Franzese et al. (2023) prove the existence of the backwards SDE. Unlike our work, these approaches make use of conditional neural fields or operate on uniform grids of coordinates, whereas approach operates raw sparse data, enabling better scaling. The closest to this work in terms of scaling is Diffusion Probabilistic Fields (Zhuang et al., 2023) which denoises coordinates independently using small coordinate subsets for context; this is much more restrictive than our approach and resolutions are much smaller than ours (up to 64 64).
7 CONCLUSION
In conclusion, we found that our anfinit-diiension Hilbert space diffusion model with transi- tion densities represented by non-local integral operators is able to generate high-quality arbitrary resolution samples. Despite only observing small subsets pixels during training, sample quality significantly surpasses prior of-the-art dimenssonnii models trained on all pixels at once. While prior comnitediimessionll approaches use latent conditional neural fields, our findings demonstrate that sparse neural operators which operate directly on raw data are a capable alternative, offering significant advantages by removing the point-wise constraint and thus the need for latent compression. Future work would benefit from improved neural operators that can effectively operate at greater levels of sparsity to improve the efficiency of our approach and enable even further scaling.
generative models, and is competitive with state-

---

REFERENCES
Michael Albergo, Nicholas Boffi, and Eric Vanden-Fijnden. Stochastic Interpolants: A Unifying Framework for Flows and Diffusions. arXiv preprint arXiv:2303.00 08797. 2023.
Ivan Anokhin, Kirill Demochkin, Taras Khakhulin, Gleb Sterkin, Victor Lempitsky, and Denis Korzhenkov. Image Generators with Conditionall-Indeeeeinn Pixel Synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14278-14287, 2021.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer Normalization. arXiv preprint arXiv:1607.06450 2016.
Lorenzo Baldassari, Ali Siahkoohi, Josselin Garnier, Knut Solna, and Maarten de Hoop. Conditional Basee-Based Diffusion Models for Bayesian Inference in Infinite Dimensions. arXiv preprint 19X1: 2023.
Jan Biemond, Reginald L Lagendijk, and Russell M Mersereau. Iterative Methods for Image Deblurring. Proceedings of the IEEE, 78(5):856-888 1990.
Christopher M Bishop and Nasser M Nasrabadi. Pattern Recognition and Machine Learning. volume 4. Springer, 2006.
Sam Bond- Bond-Taylor and Chris G Willcocks. Gradient Origin Networks. International Conference on Learning Representations, 2021.
Sam Bond-Taylor, Peter Hessey, Hiroshi Sasaki, Toby P Breckon, and Chris G Willcocks. Un- leashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Dector-Quantized Codes. In European Conference on Computer Vision, pages 170-188. Springer, 2022.
Shuhao Cao. Choose Transformer: Fourier or Galerkin. Advances Neural Information Processing Systems, 02249244-49400 2021.
Christopher Choy, Jun Young Gwak, and Silvio Savarese. 4d Spatio-Temporte ConvNets: Minkowski Convolutional Neural Networks. Proceedings the IEEE Conference on Computer Vision and Pattern Recognition, pages 3075-3084, 2019.
Spconv Contributors. Spconv: Spatially Sparse Convolution Library, https://github. Sraveller59/spcont 2022.
Giuseppe Da Prato and Jerzy Zabezyk. Stochastic Equations Infinite Dimensions. Cambridge university press, 2014.
Valentin De Bortoli, James Thornton, Jeremy Heng, and Arnaud Doucet. Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling. Advances in Neural Information Processing Systems, 34:17695-17709, 2021.
Tim Dockhorn, Arash Vahdat, and Karsten Kreis. Score-Based Generative Modeling with Critically- Damped Langevin Diffusion. In International Conference on Learning Representations, 2022.
Yilun Du, Katie Collins, Josh Tenenbaum, and Vincent Sitzmann. Learning Aignal-Agnostic Man- ifolds of Neural Fields. Advances in Neural Information Processing Systems, 34:8320-8331, 2021.
Emilien Dupont, Hyunjik Kim, SM Ali Eslami, Danilo Jimenez Rezende, and Dan Rosenbaum. From data to functa: Your data point is function and you treat like In International Conference on Machine Learning, pages 5694-5725. PMLR, 2022a.
Emilien Dupont, Yee Whye Teh, and Arnaud Doucet. Generative Models as Distributions of Functions. In International Conference on Artificial Intelligence and Statistics, pages 2989-3015. PMLR, 2022b.
Vincent Dutordoir, Alan Saul, Zoubin Ghahramani, and Fergus Simpson. Neural Diffusson Processes. arXiv preprint arXiv:2206.03399 2022.
10

---

Giulio Franzese, Simone Rossi, Dario Rossi, Markus Heinonen, Maurizio Filippone, and Pietro Michiardi. Continuous- Time Functional Diffusion Processes. arXiv preprint arXiv:2303.00800 2023.
Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J Rezende, SM Eslami, and Yee Whye Teh. Neural processes. ICML Workshop, 2018.
Paul Hagemann, Lars Ruthotto, Gabriele Steidl, and Nicole Tianjiao Yang. Multilevel Diffu- sion: Infinite Dimensional Score-Based Diffusion Models for Image Generation. arXiv preprint DrXiv:2303.04772, 2023.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by a Two Time-Scale Update Rule Converge to Local Nash Equilibrium. Advances in neural information processing systems, 30, 2017.
Dave Higdon. Space and Space- Time Modeling using Process Convolutions. In Quantitative methods for current environmental issues, pages 37-56. Springer, 2002.
Jonathan Ho, Ajay Jain, and Picter Abbeel. Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded Diffusion Models for High Fidelity Image Generation. J. Mach. Learn. Res.. 23(47): 1-33, 2022a.
Jonathan Ho, Tim Salimans, Alexey A Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video Diffusion Models. Advances in Neurul Information Processing Systems, 2022b.
Emiel Hoogeboom and Tim Salimans. Blurring Diffusion Models. In International Conference on Learning Representations, 2023.
Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. Simple Diffusion: End-to-End Diffusion for High Resolution Images. arXiv preprint arXiv:2301.1109) 2023.
Robert A Hummel, B Kimia, and Steven W Zucker. Deblurring Gaussian Blur. Computer Vision, Graphics, and Image Processing, 38(1))66-80, 1987.
Aapo Hyvärinen. Estimation of Non-Normalized Statistical Models by Score Matching. Journal of Machine Learning Research, 6(4), 2005.
Bowen Jing. Gabriele Corso, Renato Berlinghieri, and Tommi Jaakkola. Subspace Diffusion Genera- tive Models. In European Conference on Computer Vision, pages 274-289. Springer, 2022.
Fritz John. Numerical Solution of the Equation of Heat Conduction for Preceding Times. Annali di Matematica pura ed Applicata, 1001129-44, 1955.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive Growing of GANs for Im- proved Quality, Stability, and Variation. In International Conference on Learning Representations, 2018.
Tero Karras, Samuli Laine, and Timo Aila. A Style-Based Generator Architecture for Generative Adversarial Networks. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 4401-4410. 2019.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and Improving the Image Quality of StyleGAN. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 8110-8119, 2020.
Gavin Kerrigan, Justin Ley, and Padhraic Smyth. Diffusion Generative Models in Infinite Dimensions. In International Conference on Artificial Intelligence and Statistics, pages 9338-9563. PMLR,
2023.
Diederik Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In International Conference Learning Representations, 2015.
11

---

Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural Operator: Learning Maps Between Function Spaces. arXiv preprint arXi::2188.084 08481, 2021.
Alexander Kukush. Gaussian Measures in Hilbert Space: Construction and Properties. John Wiley & Sons, 2020.
Tuomas Kynkäänniemi, Tero Karras, Miik Aittala, Timo Aila, and Jaakko Lehtinen. The Role of ImageNet Classes in Fréchet Inception Distance. In International Conference on Learning Representations, 2023.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural Operator: Graph Kernel Network for Partial Differential Equations. arXiv preprint arXiv:2003.03485, 2020.
Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar, et al. Fourier Neural Operator for Parametric Partial Differential Equations. In International Conference on Learning Representations, 2021.
Jae Hyun Lim, Nikola B Kovachki, Ricardo Baptista, Christopher Beckham, Kamyar Azizzadeneshelii Jean Kossaifi, Vikram Voleti, Jiaming Song, Karsten Kreis, Jan Kautz, et al. Score- Based Diffusion Models in Function Space. arXiv preprini arXiv:2302.07400 2023.
Zhuang Liu, Hanzi Mao, Chao- Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A ConvNet for the 2020s. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 1 11976-11986 2022.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps. In Advances in Neural Information Processing Systems, 2022.
Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. Communications of the ACM, 65(1)999-106, 2021.
Hà Quang Minh. Regularized Divergences Between Covariance Operators and Gaussian Measures on Hilbert Spaces. Journal of Theoretical Probability, 34:580-643, 2021.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved Denoising Diffusion Probabilistic Models. In International Conference on Machine Learning, pages 8162-8171. PMLR, 2021.
Jakiw Pidstrigach, Youssef Marzouk, Sebastian Reich, and Sven Wang. Infinite- Dimensional Diffu- sion Models for Function Spaces. arXiv preprint arXiv:2302.10138 2023.
Francis J Pinski, Gideon Simpson, Andrew M Stuart, and Hendrik Weber, Kullback-Leibler Approx- imation for Probability Measures on Infinite Dimensional Spaces. SIAM Journal on Mathematical Analysis, 47(6):4091-4122 2015.
Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa. and Supasor Suwajanakorn. Diffu- sion Autoencoders: Toward a Meaningful and Decodable Representation. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pages 10619-10629, 2022.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas Guibas. PointNet+4: Deep Hierarchical Feature I Learning on Point Sets in a Metric Space. Advances in neural information processing systems, 30, 2017.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning Transferable Visual Models from Natural Language Supervision. In International Conference on Machine Learning. pages 8748-888 8763. PMLR, 2021.
Md Ashiqur Rahman, Zachary E Ross, and Kamyar Azizzadenesheli. U-NO: U-Shaped Neural Operators. arXiv preprint arXiv:2204 ///27, 2022.
12

---

Danilo Rezende and Shakir Mohamed. Variational Inference with Normalizing Flows. In International Conference on Machine Learning, pages 1530-1538. PMLR, 2015.
Severi Rissanen, Markus Heinonen, and Amo Solin. Generative Modelling with Inverse Heat Dissipation. International Conference on Learning Representations, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- Resolution Image Synthesis with Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-106999 2022.
David W Romero, Robert-Jan Bruintjes, Jakub Mikolaj Tomczak, Erik J Bekkers, Mark Hoogendoom, and Jan van Gemert. Flexconv: Continuous kernel convolutions with differentiable kernel sizes. In International Conference Learning Reprexentations, 2022.
Olaf Ronneberger. Philipp Fischer, and Thomas Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. In International Conference on Medical image computing and computer- assisted intervention, pages 234 241. Springer, 2015.
Tim Salimans and Jonathan Ho. Progressive Distillation for Fast Sampling of Diffusion Models, In International Conference on Learning Representations, 2022.
Rajhans Singh, Ankita Shukla, and Pavan Turaga. Polynomial Implicit Neural Representations For Large Diverse Datasets. arXiv preprint wrXiv:2301.11093, 2023.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Implicit Neural Representations with Periodic Activation Functions. Advances in Neural Information Processing Systems, 33:7462-7473. 2020.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. In International Conference on Machine Learning, pages 2256-2265. PMLR, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising Diffusion Implicit Models. In International Conference on Learning Representations, 2021.
Yang Song and Stefano Ermon. Generative Modeling by Estimating Gradients of the Data Distribution. Advances in Neural Information Processing Systems, 32, 2019.
Yang Song. Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency Models. arXiv preprint arXiv:2303.01 2023.
Haotian Tang, Zhijian Liu, Xiuyu Li, Yujun Lin, and Song Han. TorchSparse: Efficient Point Cloud Inference Engine. In Conference on Machine Learning and Systems (MLSys). 2022.
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lueas Beyer, Xiaohua Zhai, Thomas Un- terthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. MiP-Mixer: An AlI-MLP Architecture for Vision. Advances in Neural Information Processing Systems, 34: 24261-24272, 2021.
Pascal Vincent. A Connection Between Score Matching and Denoising Autoencoders. Neural computation, 23(7):1661-1674, 2011.
Clinton Wang and Polina Golland. Discretization Invariant Learning on Neural Fields. In arXiv preprint arXiv:2206.0117., 2022.
Christopher Williams, Fabian Falck, George Deligiannidis, Chris Holmes, Arnaud Doucet, and Saifud- din Syed. A Unified Framework for U-Net Design and Analysis. arXiv preprint arXiv:2305.1 19638. 2023.
Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar. Neural Fields in Visual Comput- ing and Beyond. In Computer Graphics Forum, volume 41, pages 641-676. Wiley Online Library, 2022.
13

---

Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop. arXiv preprins (.Xiv::506603365. 2015.
Bowen Zhang, Shuyang Gu, Bo Zhang, Jianmin Bao, Dong Chen, Fang Wen, Yong Wang. and Baining Guo. StyleSwin: Transformer-Base Based GAN for High-Resolution Image Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11304-1 1314, 2022.
Han Zhang, Ruili Feng, Zhantao Yang. Lianghua Huang, Yu Liu, Yifei Zhang, Yujun Shen, Deli Zhao, Jingren Zhou, and Fan Cheng. Dimensionality-Warinng Diffusion Process. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14307-14316, 2023.
Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages
2018.
Peiye Zhuang, Samira Abnar, Jiatao Gu, Alex Schwing, Joshua M Susskind, and Miguel Ángel Bautista. Diffusion Probabilistic Fields. In International Conference on Learning Representations, 2023.
A IMPLEMENTATION DETAILS
All 256 x 256 models are trained on single NVIDIA A100 80GB GPU using automatic mixed precision. Optimisation is performed using the Adam optimiser (Kingma and Ba, 2015) with a batch size of 32 and learning rate of 5 x 10 10-5, each model being trained to optimise validation loss. Each model is trained as a diffusion autoencoder to reduce training variance, allowing much smaller batch sizes thereby permitting training on single GPU, A latent size of 1024 is used and the latent model architecture and diffusion hyperparameters are the same as used by Preechakul et al. (2022). In image space, the diffusion model uses cosine noise schedule (Nichol and Dhariwal, 2021) with 1000 steps. Mollifying is performed with Gaussian blur with a variance of 1.0.
For the image-space architecture, 3 sparse residual convolution operator blocks are used the sparse data. Each of these consist of a single depthwise sparse convolution layer with kernel size 7 and 64 channels with the output normalised by the total number of coordinates in each local region, followed by three layer MLP; modulated layer normalisation (Ba et al., 2016; Nichol and Dhariwal, 2021) used to normalise and condition on the diffusion time step. These blocks use larger convolution kernels than typically used in diffusion model architectures to increase the number of coordinates present in the kernel when a small number of coordinates are sampled. Using large kernel sizes paired with MLPs has found success in recent classification models such as ConvNeXt (Liu et al., 2022).
As mentioned in Section 4.2, for the grid-based component our architecture we experimented with variety of U-Net shaped fourier neural operator (Li et al., 2021; Rahman et al., 2022) architectures; although these more naturally allow resolution changes at those levels, we found this came with a substantial drop in performance. This was even the case when operating at different resolutions. As such, we use the architecture used by Nichol and Dhariwal (2021) fixed at a resolution of 128 x 128; the highest resolution uses 128 channels which is doubled at each successive resolution up maximum factor of 8; attention is applied at resolutions 16 and 8, as is dropout, as suggested by Hoogeboom et al. (2023). Although this places more emphasis on the sparse operators for changes in sampling resolution, we found this approach to yield better sample quality across the board.
A.1 TABLE 1 EXPERIMENT DETAILS
In Table we compare against number of other approaches in terms of FIDCUP. In each case scores are calculated by comparing samples against full datasets, with 30k samples used for CelebA-HQ and 50k samples used for FFHQ and LSUN Church. For the finite-dimensional approaches we use the official code and released models in all cases to generate samples for calculating FIDeLIP scores. To provide additional context for CelebA-HQ 128x we provide scores for StyleGAN2 and StyleSwin where samples are downsampled from 256 x 256.
14

---

the official code and pretrained models for CelebA-HQ at resolutions 64 x 64 and 128 x 128; for FFHQ and LSUN Church we train models using the same generator hyperparameters used for all experiments in that work and for the discriminator the model used for CelebA-HQ 128 x 128 by adding an additional block to account for the resolution change. These models were trained by randomly sampling 1/4 of coordinates each training step to match what was used to train our models.
For GASP (Dupont et al., 2022b)
scale
Similarly, for GEM (Du et al., 2021) we use the official code and train models on CelebA-HQ 128 x 128, FFHQ and LSUN Church, using the same hyperparameters as used for all experiments in that work (which are independent of modality/resolution); we implement sampling as described in Section 4.4 of that paper by randomly sampling latent corresponding to a data point, linearly interpolating to neighbouring latent and adding a small amount of Gaussian noise (chosen with standard deviation between O and to optimise sample quality/diversity) and then projected onto the local manifold. Because GEM is based on the autodecoder framework where latents are allocated per datapoint and jointly optimised with the model parameters, training times are proportional to the number of points in the dataset. As such, the FID scores for LSUN Church in particular are especially poor due to the large scale of the dataset.
B MOLLIFIED DIFFUSION DERIVATION
In this section we derive the mollified diffusion process proposed in Section 3.2.
B.1 FORWARD PROCESS
We start by specifically choosing ( (ra) defined by Section 2.1 where states mollified by T,
(( (ro) = (1=--1 Vor- (1 TTT*)
(17)
( (m)) = Na Vo,Tuo, (1 - -
(18)
from which we wish to find the corresponding representation of q vo). The solution to this is given by Bishop and Nasrabadi (2006, Equations 2.113 to 2.117), where we write
g( (mo) = N(xri n, A A-1,
(19)
( (m)) = (11-11 Au + b, L + AA"
(20)
( [x,, mn) = (1 Av; + b, L L-
(21)
From this we can immediately see that l = voil wo and A-1 = (1 - TT* Additionally,
Ap + AvāTTz + b Vāg-1Tzz zq. therefore we can modify the approach by Ho al. (2020),
including T where relevant and set A, b and I as
A = az [1 1 a1-1) T,
18 Txo: b = 1- āi
1 - Ūt- BTT
1
(22)
1 ā
This can be shown to be correct by passing A, b, and L --1 into equations Eqs. (19) to (21), first:
Ap + b Von Varyaul Vai((1 ā 1)
Txn + b
(23)
1- αR
Vorvar - (2-1)) + v var-i(1 ai) Tw0
(24)
1 04t
Von-i(a4 - α4 +1+1 onl
T2o = Vāi-ITI0.
(25)
1- āt
and secondly,
L AA-AA-1 1 (1 - 1--;,11(11 - az)
TT + 41(1 - 11-1)2(1 - -ix) TT
(26)
1- δ2
(1 2z)²
1 - / - + ai + a: 25, + 00
(1 - 777
(27)
1 1-04
āz) - rx-1) TT = - -a--1))T+"
(28)
1 - δz
which together form q(32-1120))
15

---

B.2 REVERSE PROCESS
In this section we derive the loss for optimising the proposed mollified diffusion process and discuss the options for parameterising the model. As stated in Section 3.2, define the reverse transition densities as
po (2 12)) (z ( t), oTTT",
(29)
where of = 3e or of Be. From this we can calculate the loss at time - 1 in the same manner the finit-dimmensionl case (Eq. 3) by calculating the Kullback-Leibler divergence in infinite
dimensions (Pinski et al., 2015), for conciseness we ignore additive constants throughout since they do not affect optimisation,
C C-
T (2,, (2 t))
(30)
2
To find a good representation for we expand out iú as defined in Eq. (10) in the above loss giving
v 1 1
Var(1 (x-1)) It - mo 1) )
-.-1
Eng
T-1
Two +
(31)
2
1 1 āe
1 1 ā
From this we can see that one possible parameterisation is directly predict nu, that is,
val- (x), Var(1 - -((-1) Z1. -
po(x,,
(32)
Cte
This parameterisation is interesting because when sampling, we can use the output of fo to directly obtain estimate of the unmollified data. Additionally, when calculating the loss Lt-1, all T and 1 terms cancel out meaning there are no concerns with reversing the mollification during training, which can be numerically unstable. To see this, we can further expand out Eq. (31) using the parameterisation of defined in Eq. (32),
/a:-18 O2
V0n(1 (t-1))
Eg
T
Txo +
207
1-
(33)
Var- 1,
Var(1 -ā(-1)
Dt
1
āi
2
0
Eg
Ey
fo t)
(34)
20
āi
1 ās
H
Vo--1.30
|1 fo( 11 111
Eg
(35)
20
(z))
Using this parameterisation, we can sample from pe (2), (xx) as
Var(l (1 āt-1) Iz +
1 fo t) + 015 1-āz Oz
where {~ Ct).
(36)
1 Oz
parameterise to predict the noise rather than Tgg which was found by Ho et al. (2020) to yield higher sample quality. To see this, we can write Eq. (9) as xz(2o. Va zo + VI T. Expanding out Eq. (30) with this gives the following loss.
Alternatively,
C.
Eg
T
(((an) 20) uo(o)s))
(37)
20
H
E,g
T-1
((xr(60:E))
1 T (xr( -V1-a, - mo(
(38)
207
Vou
3r
E.
--1
x)(
TE) - vo(zert)
(39)
207
Vaz
VT - a4t
16

---

Since directly predicting & would require predicting non-continuous function, instead propose predicting Tς which is a continuous function, giving the following parameterisation and loss,
Br
(1
Zt
fo(xe,)))
(40)
Vot
vI
0t+
Br vI -1-aa
B1 vI
C Eg
fo(xe,
TE
(41)
2
de
In this case T is a linear transformation that does not affect the minima. In addition to this, we can remove the weights suggested by Ho et al. (2020), giving the following proxy loss,
Caimple
Egg [1 1) TEI
(42)
An alternative paramctcrisation which can train more reliably is e-prediction (Salimans and Ho, 2022; Hoogeboom et al., 2023); experimented with this parameterisation but found mollified noise prediction to yield higher quality samples.
C ADDITIONAL RESULTS
In this section, we provide additional details on the training speedup and memory reductions possible using our multi-scale architecture, from sparse coordinate sampling at different rates, and with different fixed resolutions of the inner U-Net. In Table 4 we calculate the training speedup for a fixed memory budget of 10GB, while in Table we calculate the memory reduction for a fixed batch size of 16. Additionally, we add here an extra quantitative result to assess sample quality at when sampling from the FFHQ model trained only on 256 256 images at a resolution of 1024 x 1024: calculating FIDCLIP on 5000 samples using 50 sampling steps, the model achieves a score of 15.84. We also provide additional samples from our models to visually assess quality. Detecting overfitting crucial when training generative models. Scores such as FID are unable to detect overfitting, making identifying overfitting difficult in approaches such as GANs. Because diffusion models are trained to optimise bound on the likelihood, training can be stopped to minimise validation loss. As further evidence we provide nearest neighbour images from the training data to samples from our model, measured using LPIPS (Zhang et al., 2018).
Data Resolution
128
256
512
Inner Resolution
32
64
64
128
128
256
2x
1.00
1.00
1.00
1.00
1.00
X
Rate
4x
1.64x
1.53x
1.97
2.08x
3.61 x
8x
x.65
2.17x
3.51
3.18x
7.98
XO
16x
2.89
2.49
4.57x
3.83x
11.78x
Table 4: Training speedup for a fixed memory budget. Larger is better. Calculated for different subsampling rates, training data resolutions, and resolutions of the grid within the multi-scale architecture.
Data Resolution
128
256
512
Inner Resolution
32
64
64
128
128
256
2x
1.56x
1.44x
1.42x
1.52x
1.43
1.32x
Rate
4x
2.44x
2.02x
1.99x
1.66x
2.00x
1.67
8x
3.27x
2.46x
2.44x
1.88x
2.44x
1.90x
16x
3.90x
2.75x
2.73x
2.01x
2.73x
2.03
Table 5: Memory reduction during training for fixed batch size. Larger is better. Calculated for different subsampling rates, training data resolutions, and resolutions of the grid within the multi-scale
architecture.
17

---

HE
Figure 10: Non-cherry picked, CelebA-HQ 256 256 samples.
18

---

aook
urra
Co
uuua i
sttak
Figure 11: Non-cherry picked, LSUN Church 256 256 samples.
19

---

Figure 12: Non-cherry picked, FFHQ 256 x 256 samples.
20

---

Figure 13: Nearest neighbours for a model trained on CelebA-HQ based on LPIPS distance. The left column contains samples from our model and the right column contains the nearest neighbours in the training set (increasing in distance from left to right)
bTi HIN
Figure 14: Nearest neighbours a model trained on LSUN Church based on LPIPS distance. The left column contains samples from our model and the right column contains the nearest neighbours in the training set (increasing in distance from left to right)
21

---

Nearest neighbours for a model trained on FFHQ based on LPIPS distance. The left column contains samples from our model and the right column contains the nearest neighbours in the training set (increasing in distance from left to right)
Figure 15:
22


Reviews:
[{"review_id": "UonpK8Ti6O", "reviewer": "Reviewer_gZH8", "summary": "In this work, the mollified diffusion models is proposed to process infinite resolution data efficiently and effectively, to overcome the weakness of low sampling speed of existing diffusion models, where the mollification of data, Fourier neural operators, multi-scale architectures and efficient sparse operators are applied. The experiments are conducted accordingly with impressive results.", "questions": "Q1. What does $x_t(x_0, \\psi)$ mean in Equation (11)? What is the difference between $x_t(x_0, \\psi)$ and $x_t$?\n\nQ2. Could the authors provide any theoretical justification and guarantee on why mollification is necessary and suitable for diffusion models to analyze infinite resolution data?\n\nQ3. Would the authors clarify what $v_0$ means above Equation (14)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. Originality: It is a novel idea to apply mollification for diffusion models  to enable the data to be processed in the Hilbert space and allow diffusion models to generate high resolution or even infinite resolution data with efficacy. Smoothing using mollification can ensure the regularity of data and enhance enable the modelling in the Hilbert space to reduce the sampling speed.\n\n2. Quality: The experimental results seem to be promising and impressive quantitatively and qualitatively. FIDs of the proposed $\\infty$-Diff are comparative to those of finite-dimensional methods. \n\n3. Significance: In this paper, regularity is discussed in diffusion models. It opens the horizon of future theoretical analysis of diffusion models.", "weaknesses": "The writing really needs to be polished. A lot of typos should be corrected, for example, Page 4 Section 3.2 Paragraph 1 Line 11 $\\mu_\\theta: \\mathcal{H}, \\mathbb{R} \\rightarrow \\mathcal{H}$ → $\\mu_\\theta: \\mathcal{H} \\times \\mathbb{R} \\rightarrow \\mathcal{H}$; Page 5 Section 4 .1  Paragraph 2 Line 1-2 “respectfully” → “respectively”; etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "G8TiMBRnBs", "reviewer": "Reviewer_RCWm", "summary": "This paper introduces an infinite-dimensional diffusion model to handle images at arbitrary resolutions. The authors introduce an architecture based on neural operators that achieves state-of-the-art FID among infinite-dimensional approaches at resolutions up to 256x256.", "questions": "- What is the \"pointwise\" constraint that is mentioned in the abstract and conclusion? Perhaps related, could the authors elaborate what is meant by \"coordinates are treated independently\" in neural fields?\n- In section 4.1, what does the notation $c \\in {m \\choose D}$ means?\n- In equation (13), I am confused by the use of both $x$ and $v_l$. I do not understand if the operator represents a function from $x$ to $s$ or $v_l$ to $v_{l+1}$.\n- I did not understand the second paragraph of section 5 on diffusion autoencoder framework. What do the authors mean by \"using the first half of our architecture\"?\n- What do the authors mean by \"each model being trained to optimize validation loss\" in Appendix A? Is validation data used to train the networks on top of training data?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "The network architecture seems to have been well designed, and the results are very convincing. In particular, the use of irregular grids seems to be a good tradeoff between efficiency and retaining fine detail information (particularly if the sampling grid varies from training point to training point, which is not clear).", "weaknesses": "- The paper lists the extension of diffusion models to infinite-dimensional spaces as one of its contributions, but there are already many works on this topic [1-5] (of which only [2-4] are briefly mentioned in the paper). The authors should discuss the relationship of their framework to this related work in greater detail (and they should be mentioned before the discussion). Also, the mathematical treatment of the extension to infinite dimensions in the paper is lacking compared to [1-5] and thus cannot be considered a contribution.\n- While the paper aims to work at any resolution, the use of \"mollification\" (I suggest to simply call this blurring which is a better-known term in the NeurIPS community) on the data distribution effectively limits the maximum resolution of the generated images. A close inspection of Figure 2 also reveals artifacts on the images generated at resolution larger than the training resolution. On a related note, the authors never discuss the mechanisms by which the score network should be expected to generate to higher resolutions while never having observed high-resolution images. Could the authors expand on this point?\n- The central contribution of the paper seems to be its architecture. While its components are described rather abstractly in Section 4, I do not understand what is being implemented exactly. The discretization of the integrals is never discussed in detail, as well as the grid on which the various intermediate activations are represented. When using a regular grid, what is the difference between the proposed architecture and a regular CNN such as UNet? As another important point, how discretization invariance is obtained when using images with different resolutions is not discussed. Does the number of points used in the finite-sum approximation of the integrals changes?\n- Super-resolution and inpainting are rather unfaithful to the original images: downsampling or cropping the generated image does not yield back the input image.\n\n[1] Kerrigan, Gavin, Justin Ley, and Padhraic Smyth. “Diffusion Generative Models in Infinite Dimensions.” arXiv, February 24, 2023. https://doi.org/10.48550/arXiv.2212.00886.\n\n[2] Lim, Jae Hyun, Nikola B. Kovachki, Ricardo Baptista, Christopher Beckham, Kamyar Azizzadenesheli, Jean Kossaifi, Vikram Voleti, et al. “Score-Based Diffusion Models in Function Space.” arXiv, February 14, 2023. https://doi.org/10.48550/arXiv.2302.07400.\n\n[3] Hagemann, Paul, Sophie Mildenberger, Lars Ruthotto, Gabriele Steidl, and Nicole Tianjiao Yang. “Multilevel Diffusion: Infinite Dimensional Score-Based Diffusion Models for Image Generation.” arXiv, April 29, 2023. https://doi.org/10.48550/arXiv.2303.04772.\n\n[4] Franzese, Giulio, Giulio Corallo, Simone Rossi, Markus Heinonen, Maurizio Filippone, and Pietro Michiardi. “Continuous-Time Functional Diffusion Processes.” arXiv, July 7, 2023. https://doi.org/10.48550/arXiv.2303.00800.\n\n[5] Pidstrigach, Jakiw, Youssef Marzouk, Sebastian Reich, and Sven Wang. “Infinite-Dimensional Diffusion Models.” arXiv, October 3, 2023. http://arxiv.org/abs/2302.10130.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Fc2TW7sYvB", "reviewer": "Reviewer_iaub", "summary": "The paper considers the generative modeling problem and the diffusion models in particular. The proposed novel diffusion model is defined in an infinite-dimensional Hilbert space in order to possibly model infinite resolution data. The model is trained only on randomly sampled subsets of coordinates and denoising data only there. It allows for learning a continuous function for arbitrary resolution sampling. Whereas the standard infinite-dimensional models use point-wise functions requiring latent compression, the proposed model employs non-local integral operators to map between Hilbert spaces and, as such, allows spatial context aggregation. The method is compared against the state-of-the-art models across different tasks achieving comparable results.", "questions": "I would like to see especially the following experiments and improvements regarding specifically to the Weaknesses section:\n1. It will be great if the authors might compare the works mentioned in the Discussion section against their model in the same experimental settings.\n2. I would like to see a new experiment comparing the computational costs of proposed model on different resolutions (e.g., in FLOPs).  \n3. The authors show the results on up to 8x subsampling rate but I’m considering what will be the highest subsampling rate (and resolution) when the model is still doing well. Please, if you could include such comparison.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper has a few significant strengths overall, which I will outline below:\n1. Proposed model achieve the best or comparable results to the state-of-the-art models.\n2. Formulating the generative diffusion model in an infinite-dimensional Hilbert space and allowing to denoise data only on a subset of coordinates is very interesting. I would assume that it might be to complex problem but maybe the mollification is why the model is able learn?\n3. Formulating the diffusion model in a Hilbert space in a strict mathematical regime, which seems to be correct.\n4. Including additional experiments on not so common tasks like inpainting.\n5. The presentation is very clear. Overall, the flow of the manuscript is well-organized.", "weaknesses": "However, despite the strengths, the paper has a few major and minor weaknesses:\n1. The works based on similar ideas are mentioned only in the Discussion section, but the proposed model is not compare against them in an experimental way. \n2. The lack of information regarding the computational costs of both training and inference for the proposed model. \n3. I couldn’t find the information which is actual upscaling procedure limit, which will be significant from the practitioner point of view.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6Cl5jxEWTJ", "reviewer": "Reviewer_btvx", "summary": "**Edit: I have raised my score to 6 to reflect the author's updates to the submission.**\n\nThis paper proposes a generalization of diffusion models for data living in infinite dimensional Hilbert spaces. The main motivation for doing so is to enable models which can be trained and sampled at arbitrary resolutions and to enable scaling of diffusion models to high-dimensional data. Section 3 discusses some theoretical concerns for developing such models, Section 4 proposes an architecture well-suited for this task on natural image data, and Section 5 provides an empirical evaluation of the proposed methodology.", "questions": "See weakness section.\n\nOverall, I think this work provides an important and novel practical contribution (namely an empirically sound architecture for infinite-dimensional diffusions), but the contributions regarding the theory are over-stated and the framing of the work with regards to prior work in this area needs to be significantly improved.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The proposed neural architecture is novel and a significant contribution to the literature on infinite-dimensional models. In particular, the proposed model shows significant performance gains in comparison to several previously proposed infinite-dimensional generative models (Table 1) in terms of FID scores.\n- The experiments are generally well-executed and convincing in terms of the conclusions drawn from these throughout the paper.", "weaknesses": "- A major weakness of this paper is how the authors frame their contributions within the existing literature. \n     - For example, in the abstract the authors write \"Unlike prior infinite-dimensional models, which use point-wise functions requiring latent compression, our method employs non-local integral operators to map between Hilbert spaces, allowing spatial context aggregation.\" Similarly, in Section 1, the authors write \"We introduce a new Gaussian diffusion model defined in an infinite-dimensional state space that allows infinite resolution data to be generated (see Fig. 2)\". However, numerous recent works have posed infinite-dimensional diffusion models which precisely use these integral operators and a similar theoretical framework [1, 2, 3, 4, 5, 6, 7]. \n     - The authors are clearly aware of this work (see Section 6) but do not appropriately frame their contributions, i.e. many of these prior works develop the theory of infinite-dimensional models which is very closely related to the proposed theory, but the authors of this submission do not state this. The authors claim that these works are concurrent, but the earliest of these works appeared in December 2022 [4], February 2023 [1], and March 2023 [2, 3], more than 6 months prior to the ICLR submission deadline. In addition, the authors are missing a citation to Kerrigan et al. [4] who previously develop a theory for discrete-time diffusion models which is closely related to the theory the authors propose in Section 3, as well as references to several other infinite-dimensional diffusion models in continuous time [5, 6, 7].\n\n- The theory in Section 3 is imprecise to the degree of incorrectness. \n     - The authors write \"The Radon-Nikodym theorem tells us the density for a measure $\\nu$ absolutely continuous with respect to a base measure $\\mu$\". This is not what the Radon-Nikodym says -- the Radon-Nikodym theorem states the *existence* of a density given the absolute continuity. To actually compute this density, you require stronger results, such as the Cameron-Martin and Feldman-Hajek theorems. See [1, 4] for a discussion of these theorems in the context of diffusion models.\n     - The authors miss key regularity assumptions necessary on the Gaussian noise in order to obtain a well-posed model, see again [1, 4] for a discussion of these requirements.\n\n\n### Minor\n- The discussion of the \"diffusion autoencoder framework\" in Section 5 could use a short description for those unfamiliar with the work, i.e. it was not entirely clear to me how model is trained based on the description provided alone.\n\n\n### References:\n[1] [Lim et al., Score-Based Diffusion Models in Function Space](https://arxiv.org/abs/2302.07400)\n\n[2] [Franzese et al., Continuous-Time Functional Diffusion Processes](https://arxiv.org/abs/2303.00800)\n\n[3] [Hagemann et al., Multilevel Diffusion: Infinite Dimensional Score-Based Diffusion Models for Image Generation](https://arxiv.org/abs/2303.04772)\n\n[4] [Kerrigan et al., Diffusion Generative Models in Infinite Dimensions](https://arxiv.org/abs/2212.00886)\n\n[5] [Lim et al., Score-based Generative Modeling through Stochastic Evolution Equations\n](https://neurips.cc/virtual/2023/poster/72191)\n\n[6] [Baldassari et al., Conditional score-based diffusion models for Bayesian inference in infinite dimensions](https://arxiv.org/abs/2305.19147)\n\n[7] [Pidstrigach et al., Infinite-Dimensional Diffusion Models](https://arxiv.org/abs/2302.10130)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)