Arbitrarily-Conditioned Multi-Functional Diffusion for Multi-Physics Emulation
Authors: Da Long
Zhitong Xu
Guang Yang
Akil Narayan
Shandian Zhe
Venue: ICML 2025 poster (2025)
Keywords: Surrogate Model
Multi-physics
Diffusion Model
Gaussian Process
Functional Diffusion
Primary area: applications->chemistry_physics_and_earth_sciences

Abstract: Modern physics simulation often involves multiple functions of interests, and traditional numerical approaches are known to be complex and computationally costly. While machine learning-based surrogate models can offer significant cost reductions, most focus on a single task, such as forward prediction, and typically lack uncertainty quantification --- an essential component in many applications. To overcome these limitations, we propose Arbitrarily-Conditioned Multi-Functional Diffusion (ACM-FD), a versatile probabilistic surrogate model for multi-physics emulation. ACM-FD can perform a wide range of tasks within a single framework, including forward prediction, various inverse problems, and simulating data for entire systems or subsets of quantities conditioned on others. Specifically, we extend the standard Denoising Diffusion Probabilistic Model (DDPM) for multi-functional generation by modeling noise as Gaussian processes (GP). 
We propose a random-mask based, zero-regularized denoising loss to achieve flexible and robust conditional generation. We induce a Kronecker product structure in the GP covariance matrix, substantially reducing the computational cost and enabling efficient training and sampling. We demonstrate the effectiveness of ACM-FD across several fundamental multi-physics systems.

Arbitrariy--onddiionee
Multi-Functional Diffusion for Multi-Physics Emulation
Da Long' Zhitong Xu Guang Yang*
Akil Narayan 23 Shandian Zhe
Abstract
approaches (Zienkiewicz et al., 1977; Mitchell & Griffiths, 1980), while offering strong theoretical guarantees, often complex to implement and computationally expensive to run. In contrast, machine learning based surrogate models - also known as emulators - are trained on simulation or measurement data and can significantly reduce computa- tional costs, making them a promising alternative (Kennedy & O' H'Hagn, 2000; Razavi al., 2012).
Modern physics simulation often involves multi- ple functions of interests, and traditional numer- ical approaches are known to be complex and computationally costly. While machine learning- based surrogate models can offer significant cost reductions, most focus on single task, such as forward prediction, and typically lack uncer- tainty quantification - an essential component in many applications. To overcome these limita- tions, we propose Etbiraaily-Cooddtionne Multi- Functional Diffusion (ACM-FD), versatile prob- abilistic surrogate model for multi-physies emu- lation. ACM-FD can perform wide range of tasks within a single framework, including for- ward prediction, various inverse problems, and simulating data for entire systems or subsets of quantities conditioned on others. Specifically, we extend the standard Denoising Diffusion Proba- bilistic Model (DDPM) for multi-functional gen- eration by modeling noise Gaussian processes (GP). We propose a random-mask based, zero- regularized denoising loss to achieve flexible and robust conditional generation. We induce Kro- necker product structure the GP covariance ma- trix, substantially reducing the computational cost and enabling efficient training and sampling. We demonstrate the effectiveness of ACM-FD across several fundamental multi-physics systems. The code is released at https: / / g thub. com/ BayesiannI Inw.aaan
However, modern physical simulations often involve multi- ple functions of interest, such as initial and boundary con- ditions, solution or state functions, parameter functions, source functions, and more. Current machine learning-basce surrogate models, such as neural operators (Li et al., 2020a; Lu al., 2021: Kovachki et al., 2023), primarily focus on single prediction task, for instance, forward prediction of the solution function. To perform other tasks, one typ- ically needs to retrain a surrogate model from scratch. In addition, most existing methods do not support uncertainty quantification, which is important in practice. For example, confidence intervals are important to assess the reliability of emulation results, and in inverse problems, a posterior dis- tribution required since such problems are often ill-posed.
To address these limitations, Conditioned Multi-Functional Diffusion (ACM-FD), ver- satile probabilistic surrogate model for multi-physics emu- lation. Within a single framework, ACM-FD can handle a wide range of tasks, including forward prediction with dif- ferent input functions, various inverse problems conditioned different levels of information, simulating data for entire systems, and generating subset of quantities of interest conditioned on others. As generative model, ACM-FD produces predictive samples, naturally supporting uncer- tainty quantification across all contexts. The contributions of our work are summarized as follows.
propose Arbitrarily-
Introduction
Multi-Functional Diffusion Framework: We pro- pose multi-functtonal diffusion framework based on the Denoising Diffussion Probabilistic Model (DDPM) (Ho et al., 2020). By modeling the noise as multiple Gaussian processes (GPs), we perform dif- fusion and denoising in functional spaces, enabling the generation of multiple functions required in multi- physies systems.
Physical simulation plays a crucial role in numerous scien- tific and engineering applications. Traditional numerical
'Kahlert School of Computing, University of Utah 'Department of Mathematics, University of Utah "Sientiific Computing and Imaging Institute, University Utah. Correspondence to: Shan- dian Zhe zzhe@cs.utah.cdus.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Innovative Denoising Loss: We introduce denois-

---

Arbitrarily-onndtionn Multi-Functional Diffusion for Multi-Phhyses Emulation
ing loss that encapsulates all possible conditional parts within the system. During training, we repeatedly sam- ple conditional components using random masks, train- ing the denoising network not only to restore noise for the parts to be generated, but also to predict zero values for the conditioned components. This regular- ization stabilizes the network and prevents excessive perturbations in the conditioned components during generation. This way, ACM-FD can flexibly generate function values conditioned on any given set of func- tions or quantities, allowing it to tackle wide range of tasks, including forward prediction, inverse inference, completion, and system simulation.
where ((x)) represents the original data distribution, and each transition x(x,)x+-1) = (x v1 1,T with 3r > 0 as the noise level at step t. From this, it is straightforward to derive the relationship between the noisy instance x, and the original instance Xo.
X = V / + V1 - T.St, E. ((11,D (2)
where â. = I13- Oj and 0r = 1- 3r As increases, āg approaches zero, and 1 - a approaches 1, indicating that x; is gradually converging to a standard Gaussian ran- dom variable. When becomes sufficiently large, we can approximately view x, Gaussian white noise.
Efficient Training and Sampling: To enable efficient training and sampling, we use a multiplicative kernel to induce a Kronecker product structure within the GP covariance matrix. By leveraging the properties of the Kronecker product and tensor algebra, we bypass the need to compute the full covariance matrix and its Cholesky decomposition, substantially reducing the training and sampling costs.
DDPM then learns to reverse this diffussion process to re- construct the original instance X, from the Gaussian white noise. Data generation, or sampling, is achieved by running this reversed process, which is often referred to as the de- noising process. The reversed process is modeled as another chain, expressed as
pe (xT,,....... ...,, .x) p(xx)) II. p
(3) where p(XT) = _N I), and Q denotes the model parameters. DDPM uses the variational learning frame- work (Wainwright et al., 2008), which leads to matching each transition pe (xx) to the reversed conditional dis- tribution from the forward diffusion process,
Experiments: We evaluated ACM-FD on four fun- damental multi-physies systems, with the number of involved functions ranging from three seven. In twenty-four prediction tasks across these systems, ACM-FD consistently achieved top-tier performance compared to state-of-the-art neural operators specifi- cally trained for each task. Furthermore, we evaluated ACM-FD emulating all functions jointly and fune- tion completion. The data generated by ACM-FD not only closely adheres the governing equations but also exhibits strong diversity. Its quality is comparable to that of a diffusion model trained exclusively for un- conditional generation. In parallel, ACM-FD achieves substantially higher completion accuracy than popular inpainting and interpolation methods. ACM-FD also provides superior uncertainty calibration compared to alternative approaches. Finally, series of ablation studies confirmed the effectiveness of the individual components of our method.
q 1x Xo)
1
B. V1
=N x
X;
Ee
(4)
/1
Be
1- 3r. Accordingly, DDPM employs a neural network @ to approximate the noises & see (2). The training objective to minimize the denoising loss, Ed] (x(x,,)) t)
where 3r
I-a,
3 Method
We consider multi- multi-physics system involving a collection of M functions. These functions may represent initial and boundary conditions, source terms, parameter functions, system states and others. Our goal is to capture the under- lying complex relationships between these functions and perform a wide range of prediction and simulation tasks, such as forward prediction of the system state given the initial condition, inverse inference about the systems pa- rameters given observed states, joint simulation of all the functions, or conditional simulation of one set of functions based on another.
2 Preliminaries
The denoising diffusion probabilistic model (DDPM) (Ho et al., 2020) is one of the most successful generative models. Given a collection of data instances, such as images, DDPM aims to capture the complex underlying distribution of these instances and generate new samples from the same distribu- tion. To achieve this, DDPM specifies forward diffusion process that gradually transforms each data instance Xo into Gaussian white noise. The forward process is modeled Gauss-Markon chain,
Many of these tasks involve mapping between functions, making neural operators (Li et al., 2020a; Lu et al., 2021; Kovachki et al., 2023) - a recently developed class of sur- rogate models - an appealing approach. However, most
g(xo. ..., xT) = =x(x) II, g(xx(xx-1),
2

---

Arbitrarit-Conddtione Multi-Functional Diffusson for Multi-Phyyses Emulation
neural operators are trained for a single prediction task. To perform different task, one would need to train another neural operator from scratch. These models generally lack uncertainty quantification, which is important for many ap- plications. Furthermore, these models are unable to carry out data generation.
and substitute the neural network's prediction for &, in (8)
to obtain (ff for denoising and generation.
Next, we generalize to the fulltiofunctional case. Given M functions of interest, F = {fo (-).... fo (.)}, we employ the same diffusion process as specified in (5) and (7) to transform each function into a noise function. Next, we use a single denoising neural network to jointly recover or sample all the functions, thereby capturing the complex and strong relationships among them. Specifically, we construct network te that takes the step (, all the noisy function values sampled at t. and the sampling locations as the input, to predict the corresponding noises,
To address these issues, we propose ACM-FD, a new gen- erative model that can serve as a powerful and versatile probabilistic emulator for multi-physice systems.
3.1 Funtio-unctional Diffusion
We first extend DDPM to enable functional data generation. To this end, we generalize the diffusion process in (2) to the functional space. Given a function instance fu(-), we gradually transform it into a noise function via
(e F,M. .t 2) == (E!..... E,M).
(9) (-) is the noisy
where each r/ = (2x), E = version of fo at step 1, (.) is the corresponding noise function that corrupts fo to yield f. Zg are the sampling locations of for and Z = {2x} MM
fe(+) = vārf(() + ax))),,
(5)
where fe(-) is the noisy version of fo at step t, and Ec(.) is the noise function used to corrupt fo- To model the noise function, one can employ an expressive stochastic process. We choose to sample each noise function from a zero-mean Gaussian process (GP) (Rasmussen & Williams, 2006),
3.2 Arbitrarily- Denoising Loss
The design of our multi-functionn denoising model (9) en- ables us to perform a wide range of conditional sampling and prediction tasks, beyond just functional data generation.
&
z(),z'),
Specifically, let us denote an arbitrary set of conditioned (6)
function values as F = U (Zz) E C). and the target function values to be generated as F {fo (22)) S}. Here C and S denote the indices of the functions to be conditioned on and those to be generated, respectively. Note that and S may overlap. provided that for any k E Cns, the sampling locations do not, i.c., zgnzZ = 0. To generate F conditioned on F we fix F in the input to the network De as constant, and vary only the noisy state associated with F*, denoted as Fū (at step f). The input to & consists of F UF; (t. 2}. We use the predicted noise corresponding to Fÿ to sample F;-1, following the denoising process. This procedure is repeated until f. = O. resulting in a sample for F The sampling process is summarized in Algorithm 2.
where k((::) :) is the covariance (kernel) function and and z' denote the input locations of the function. When becomes sufficiently large (i.e.. âe 2 0), we can approximately view fe ~ (P(( x(2, z)), meaning it effectively turns into a noise function.
The actual data is the function instance sampled at a finite set of locations, f, = fo(Z)- = (Fu(z)).... So(z,))
where 2 = [2,[1 S j < N} are the sampling locations. Therefore, we only need to apply the diffusion process (5) to Z (with the function values at all the other locations marginalized out), which yields:
f, = Vofo + VI- & ~(~,)-),
By varying the choices of F and F our model can perform wide variety of data generation and prediction tasks. When F = 0, the model jointly samples the M target functions at the specified input locations. When F * 0 and C = 0. the model generates one set of functions conditioned on the other, supporting various forward prediction and inverse inference tasks. For instance, by setting C to include initial conditions or current and/or past system states and source functions, and S to the future system states, the model performs forward prediction. Alternatively, if the observed system states are included in C and S corresponds to initial/boundaryy conditions, system parameters, or past states, the model carries out inverse inference. Even for the same type of tasks, the functions and the number of functions in C, as well as the number of sampling locations
where f. = f2)) & = (Z), and K- k(Z, Z) is the covariance matrix Z.
Based on (7), can derive the reversed conditional distri- bution,
((fr- fo)
(8)
3. VI
=N (f-1l
f;
E1 Cze
3.K
VT-
3e
Comparing to (4), the only difference is that the identity ma- trix is replaced by the covariance matrix K at the sampling locations Z. We then use a similar strategy as in DDPM. We train a neural network & such that & 2) x E
3

---

Arbitraril--Cnndttonee Multi-Functional Diffusion for Multi-Physscs Emulation
(ct) ~ gP())
Random Mask (h*)
Generation Part
(6))
Timestep t
Denoising Network
Corrupted by (C2 }
Conditioned Part
210
{CE)
G.0
Figure 1.
The illustration of ACM-FD. The grey shaded area represents zero values introduced by random mask. The denoising network is trained to predict zeros for the conditioned part while recovering noises for the part to be generated (i.e., unshaded noises).
in Zg. can be adjusted to accommodate different levels of available information for prediction. When C n S A 0, the model performs not only conditional sampling of other functions but also completion of the observed functions at new locations. All tasks can be executed using a single model, Pe. during inference, following the same sampling process. One can generate many predictive samples as needed, naturally supporting uncertainty quantification in arbitrary context. Thereby, our method offers versatile probabilistic surrogate for multi- multi-physies emulation.
producing excessively large perturbations in the conditioned part. This encourages the model to better adhere to the system's 's underlying mechanism (e.g., governing equations) during generation. See Fig. for an illustration.
The choice of p(H) is flexible. We may randomly mask individual observed values within each function and/or randomly mask entire funetions. In the absence of prior knowledge specific preferences, natural choice is to set the masking probability to 0.5, as it yields the maximum masking variance (i.e., diversity).
Our de- noising loss (10) accommodates all possible conditional components for conditioned sampling as well as uncondi- tioned sampling (i.e., H consists of all zeros), allowing our model be trained for all kinds of tasks. We use stochastic training, which involves randomly sampling the mask H. applying it to the function and noise instances as specified in (11), and computing the stochastic gra- dient Ve
However, to build well-perfomming model, training to predict all the noises as specified in (9) is far from sufficient, because it only handles unconditioned sampling. To train our model to handle all kinds of tasks, we introduce a ran- dom mask variable, 7 = (h', ..... h W. Each element in H is binary, indicating whether the corresponding function value is conditioned or to be sampled. If function value is conditioned, then it should remain fixed as a constant input to L, and correspondingly, the predicted noise for that value should be zemo, since is never corrupted by any noise. Accordingly, we introduce a new denoising loss,
P.M 1, 2), @ ....... ) date the model parameters O.
|| up-
((()
(10)
3.3
Efficient Training and Sampling
= =E,Ep where for each 1 < k $ M,
R,M,,, D (f....... ....,, ,4, 2) - (8) (2M)
The training and generation with our model requires repeat- edly sampling noise functions from GPs, as specified in (7). The sampling necessitates the Cholesky decomposition of the covariance matrix K at the input locations, which has time complexity of O(N3), where N the number of input locations. When N large, the computation becomes prohibitively expensive even infeasible. However, large values of N is not uncommon in realistic systems. For in- stance, generating 2D) function sample on a 128 128 mesh results in N = 128 x 128- = 384.
p = t h + P o 0(1 - (1-h*),
hh). 0 o h² t
(11)
Here f is the original function instance sampled the spec- ified locations 2 F. Vä,f + /1 a is the noisy version of E at step and denotes the clement-wiss wise prod- uct. From (11), we can see ? is mixed with zeros for the conditioned part and the noises for the part generate. The zeros in ? regularizes the network Po, preventing it from
To address this challenge, we use a multiplicative kernel to model the noise function. Given the input dimension D, we
4

---

Arbitrarit-Conddtinne Multi-Functional Diffusion for Multi-Phyysiss Emulation
construct the kernel with the following form,
namely K _ x (2 2x) = K2 @ & KD where each = k 7a). We then perform the Cholesky decompo- sition on each local kernel matrix, yielding K, = L Utilizing the properties of Kronecker products. we can de- rive that K where A Li @ ... L Consequently, to generate sample of & on the mesh, we can first sample a standard Gaussian variable 1y (O,,,),, I), and then obtain the sample as vec(E, A 1) where veo(-) denotes the vectorization. Note that &, is a mi x... x mn tensor. However, directly computing the Kronecker product in A and then evaluat- ing A 7) can be computationally expensive, with a time complexity of O ), where me II my represents the total number of the mesh points. To reduce the cost, we use tensor algebra (Kolda, 2006) to compute the Tucker prod- uet instead, which gives the same result without explicitly computing the Kronecker product,
<=(2, 2
I d=1 (/)))//))
(12)
Notably, the widely used Square Exponential (SE) kernel already exhibits this structure: KsE(2, x') =
(L, L @...& & (ID')
" = A
z' 2/122/1)) We position the sampling locations for each tar- get function (-) on an m. x... X my mesh, denoted as Z V/2 x... ... x Y/D where x denotes the Cartesian product, and each Ya (1 I S D) comprises the m. input locations within dimension d. From the multiplicative kernel (12),
Algorithm 1 Training ..... ZM, ((H)
1: repeat
2: Sample instances of the M functions of interest over
meshes| { denoted by (f)....,
3: f ~~ Uniform(1,....
4: Sample M noise functions from GPs over (2 1 denoted by (E)...... M.M), where each & corre- sponds to flf using (12) and (13)
(13) where we reshape into an mi X ... X mD tensor II, and X is the tensor-matrix product along mode k. The time complexity of this operation becomes (( ma)).
E = 11 L, X2... ... XD LD
5: Sample the mask H = (h+ ...., hd } ~ p(H)
6: Take gradient descent step
Overall, this approach avoids computing the full covariance matrix K for all the mesh points along with its Cholesky de- composition, which would require O(m²) time and ((m)) space complexity. Instead we only compute the local ker- nel matrix for each input dimension. For example, when using 128 x 128 mesh, the full covariance matrix would be 128 x 128 which is highly expensive to compute. In contrast, our approach requires the computation of only two local kernel matrices, each of size 128 x 128. Our approach further uses the Tucker product to compute the noise func- tion sample (see (13)), avoiding explicitly computing the ex- pensive Kronecker product. As a result, the overall time and space complexity is reduced to O mi + m ma)) and O m² + m.) respectively. This enables efficient training and generation of our model. Finally, we summarize our method in Algorithm I and 2.
Ve
Fall
P. 1, 3) - C
ZM
where z - (2x) ) each P/ and - (1 S k S M)
are masked instances as defined in (11).
7: until Converged
Algorithm 2 Generation (conditioned: FS, target: F", target locations: Z' all locations: Z {2x})
1: Sample noise functions from GPs over Z using (12) and (13), denoted as E
2: Fÿ- < Subset of g at z
3: for = T....,, do
4: E+0
4 Related Work
5: if > then
6: Re-sample & following STEP 1
Generative modeling is fundamental topic in machine learning. The DDPM (Ho et al., 2020) is recent a break- through, and numerous subsequent works have expanded upon this direction, further advancing the field, such as score-based diffusion via SDEs (Song et al., 2021b), denois- ing diffusion implicit models (DDIM) (Song et al., 2021a), and flow-matching (Lipman al., 2022: Klein et al., 2023). Recent works have explored diffusion models for inverse problems, such as (Chung et al., 2023; Wang et al., 2023a; Song et al., 2023). However, these approaches mainly train an unconditional diffusion model, and then perform condi- tional sampling at the inference stage.
7: E+ Subset of E at Z
8:
end if
L F U Fj,t, 10: Subset of E, at Z
9:
11:
Generate sample
B, E
Fi
F2
v B.
VI
B,
V1
12:
end for 13: return F
we can induce a Kronecker product in the covariance matrix,
A few recent works have explored diffusion models for
5

---

Arbitrariy--onndtioeed Multi-Functional Diffusson for Multi-Phyyses Emulation
function generation, such as (Kerrigan et al., 2023) based on DDPM, (Lim al., 2023; Franzese al., 2024) lever- aging score- or SDE-based diffusion, (Zhang & Wonka, 2024) using DDIM, and (Kerrigan et al., 2024) with flow matching. Our approach, which incorporates a GP noise function into the DDPM framework, is conceptually sim- ilar to (Kerrigan et al., 2023), where a GP is treated as a special case of a Gaussian random element in their formu- lation. However, the key distinction lies in the scope of our work. Unlike prior methods focused on single-function generation, our approach is designed for fulti-aunntional scenarios, simultancously addressing diverse single- and multi-functional generation tasks conditioned on varying quantities and/or functions during inference. To achieve this, we propose an arbitrarily conditioned multi-functional diffu- sion framework that integrates a random masking strategy, a reto-regulrr denoising loss, and an efficient training and sampling method. Our work is related to (Gloeckler et al., 2024), which leverages score-based diffusion for simulation and also uses a random masking strategy to fulfill flexible conditional sampling. But (Gloeckler et al., 2024) does not enforce zero regularization on the score model's predictions for the conditioned part. In contrast, our method incorpo- rates zero regularization in the loss function to prevent the denoising network from producing excessive artificial noise to the conditioned part, thereby enhancing stability. A de- tailed comparison is provided in Section 5. Another related work is (Wang et al., 2023b), which introduces a score-based diffusion model for multi-fidelity simulation. However, the approach is not formulated as function generation and is limited to single prediction task.
tem that involves three functions: the permeability field
a. the flow pressure u, and the source term f. ID convection- diffusion system that involves three spatial- temporal functions: the scalar field of the quantity of interest u, the velocity field v, and the source function s.
Convection Diffuusion (C-D):
Diffusion Reaction (D-R): a 2D diffusion-reaction system, for which we are interested in four spatial functions: fi and f2, which represent the activator and inhibitor functions at time & = 2.5, and ii and u2. denoting the activator and inhibitor at / 5.0.
Torus Fluid (T-F): a viscous, incompressible fluid in the unit torus, for which we are interested in seven functions, including the source function f. the initial vorticity field Wo. and the vortocity fields W1...... Ws five different time steps t 2, 4, 6, 8, 10.
The details about these systems, along with the preparation of training and test data, are provided in Appendix Section A.
5.1 Predictive Performance Across Various Tasks
We first examined performance of ACM-FD across a vari- ety of prediction tasks, each predicting one function using another set of functions. These tasks address wide range of forward prediction and inverse inference problems. For example, in the case of Darcy predicting the pressure field given the permeability field a and the forcing term f is a forward prediction task, while predicting from u and f. or predicting f from u and a, typically represents in- verse problems. Due to the versatility of ACM-FD, a single trained model can be used to perform all these tasks.
Neural operator (NOs) aim to learn function-to-function mappings from data, primarily for estimating PDE operators from simulation data and performing forward predictions of PDE solutions given new inputs (e.g., parameters, initial and/or boundary conditions, source terms). Important works include Fourier Neural Operators (FNO) (Li et al., 2022) and Deep Operator Net (DONet) (Lu et al., 2021), which have demonstrated promising prediction accuracy across various benchmark PDEs (Lu et al., 2022). Many NO models have been developed based on FNO and DONet, such as (Gupta et al., 2021; Wen et al., 2022; Lu et al., 2022; Tran et al., 2023). Li et al. (2024) developed an active learning method to query multi-resolution data for enhancing FNO training while reducing the data cost. Recent advances in kernel operator learning have been made by Long et al. (2022), Batlle et al. (2023), and Lowery et al. (2024).
We compared with the following popular and mate-offthe- art operator learning methods: (I) Fourier Neural Operator (FNO) (Li et al., 2020b), which performs channel lifting, followed by a series of Fourier layers that execute linear functional transformations using the fast Fourier transform and then apply nonlinear transformations, ultimately culmi- nating in a projection layer that produces the prediction. (2) Deep Operator Net (DON) (Lu et al., 2021), which employs branch-net and a trunk-net to extract representations of the imput functions and querying locations, producing the prediction by the dot product between the two representa- tions. (3) PODDON (Lu et al., 2022), a variant of DON where the trunk-net is replaced by the POD (PCA) bases ex- tracted from the training data. (4) GNOT (Hao et al., 2023), transformer-based neural operator that uses cross-attention layers to aggregate multiple input functions' information for prediction. We utilized the original implementation of each competing method. In addition, we compared with (5) Sim- former (Gloeckler et al., 2024), an attention based emulator which also uses a random masking strategy to fulfill flexible conditional generation. However, Simformer does not fit
5 Experiments
For evaluation, we considered four fundamental multi- physics systems:
Darcy Flow (D-F): a single-phase 2D Darey flow sys-
6

---

Arbitrarity-Condttoned Multi-Functional Diffusion for Multi-Phyyscs Emulation
Table 1. Relative L error for various prediction tasks. The results were averaged over five runs.
Dataset
(ss
ACM-FD
FNO
GNOT
DON
Simformer
f. u a
1.32e-02 (2.18e-04)
1.88e-02 (1.66c-04)
1.35c-01 (6.57e-05)
2.38c-02 (3)45e-04)
1.18c-01 (3.00e-03)
a. u to f
1.59e-02 (1.59e-04)
2.37c-02 (1.87c-04
1.00++00 (0.00e+00)
3.76c-02 (7.75e-04)
4.1le-02 (2.87c-03)
D-F
a f to u
1.75e-02 (4.16e-04)
6,29c-02 (4. 18c-4))
6.09c-01 (2.40e-01)
6.05e-02 (7.17c-04)
4.04c-02 (5.17c-03)
to
3,91c-02 (7.08c-04)
5.57e-02
(4.16e-04)
1.35e-01 (1.99e-04)
5.08e-02 (5.91e-04)
1.44c-01 (4.23e-03)
u too f
3.98e-02 (6.45e-04)
5.50e-02
(5.47e-04)
9.99e-01
(7.48e-04)
6.46e-02 (1.13c-04)
1.06c-01 (3.98e-03)
8, u to v
2.17e-02 (4.53e-04)
4.50e-02
(3.89e-04)
3.26e-02 (3.41e-03)
3.64e-02 (5.07=004)
3.96e-01 (4.79e-02)
1, u to 8
5,45e-02 (1.40e-03)
7.93c-02
(8.48c-04)
1.22c-01
(1.91c-03)
(7.53c-04)
5.76c-02 (7. (7.10c-02)
C-D
8,8 M
1.60c-02 (2.15c-04)
7.26c-02
(2.16c-04)
5.80e-03
(1.51e-04)
7,86c-02 (7.42e-04)
1.03c-01 (1.95e-02)
to
2.66e-02 (3.08e-04)
5.90e-02
(8.22e-04)
6.69e-02
(3.66e-03)
4.55e-02 (6.09e-04)
5.108-01 (7.56e-02)
to s
6.06e-02 (2.54c-04)
1.16e-01 (5.63e-04)
1.85e-01 (2.84e-03)
9.65e-02 (5.52e-04)
9.21e-01 (1.00e-01)
f1, u1 to f2
1.44c-02 (8.96c-04)
1.07e-02 (1.92e-044
4,53c-01
(4.34c-02)
2.93c-01 (1.29c-03)
3.39c-02 (2.97c-03)
fi, to
1.59e-02 (3.68e-04)
2.02c-02
(2.42c-04)
3.91c-01 (1 (1.86c-02)
2.03e-01 (2.22c-03)
3.67c-02 (2.36c-03)
D-R
f2, u2 to
4.10c-02 (8.93c-04)
5.52e-02
(3.01e-03)
6.53e-01 (2.04e-02)
4.24e-01 (9.26e-04)
1.21e-01 (3.1le-03)
f2. ug to un
5.86c-02 (3.43c-04)
7.82e-02 (1.29e-04)
4.88e-01 (2.92e-02)
2.98e-01 (2.61e-03)
1.01e-01 (2.70e-03)
wo, Ms to W1
2.73c-02
(4.78c-03)
1.28e-02
(2.38e-04)
2,40c-02 (8.74c-04)
6.32c-02 (2.72c-04)
6.
6.14c-02 (2.44c-03)
Wo. wa to W2
2.43e-02
(1.60e-03)
2.08c-02 (9.80c-05)
4.00e-02 (5.92e-04)
7.69e-02 (4,41e-04)
6.99e-02 (2.18e-03)
wo; wa wa 2.43e-02
(3.17e-03)
2.33e-02 (1.83-004)
4.74e-02 (1.23e-03)
7.34e-02 (2.88e-04)
8.34e-02 (2.60e-03)
Wo, uh to Wia 1.68e-02
(1.81e-03)
1.41c-02 (1.17c-04)
3,95e-02 (6.73e-04)
5.57e-02 (1.73e-04)
9.75e-02 (3.93e-03)
Wo, Wt to f
1.63e-02
(1.49e-03)
1.79e-02 (3.04e-04)
5.91e-02 (4.01e-03)
4.77c-02 (5,56e-04)
1.14e-01 (4.00e-03)
T-F
Wo, f to wi
3.10e-02
(4.08e-03)
9.68e-03
(3.22e-04)
2.09e-02 (3.62e-04)
6.08e-02 (3.14e-04)
6.06e-02 (2.03e-03)
wo. f to w2
3.28e-02
(4.79c-03)
1.70e-02
(3.51e-04)
4.15e-02 (8.21e-04)
7.73e-02 (8.18e-04)
6.18c-02 (1.02e-03)
wo, f to w'
3.49c-02
(2.38e-03)
2.38e-02 (8.37e-05)
5.61e-02 (8.23e-04)
8.82e-02 (4.45e-04)
5.67c-02 (1.83-003)
wo, f to uh
3.34c-02
(3.87e-03)
3.10e-02
(1.26e-04)
6.97c-02
(1.62e-03)
1.02c-01
(7.28e-04)
4.10e-02 (1.98e-03)
wo, f to W75
3.26e-02 (2.13e-03)
3.81c-02 (2.01c-04)
8,35e-02 (7.33e-04)
1.21e-01 (8.20c-04) 1.18c-01 (4.15e-03)
zero values for the conditioned part as in our model; see Equations (10) and (11). The original Simfomer was devel- oped with score-based diffusion (Song et al., 2021b). For fair comparison, we adapted Simformer to the DDPM frame- work, and reimplemented it using Py PyTorch. Our method, ACM-FD, was implemented with Py Torch as well. For GP noise function sampling, we used the (SE) kernel. In all the experiments, we used FNO to con- struct our denoising network &e. Thereby, our comparison with FNO can exclude the factors from the architecture de- sign. We combined two types of random masking: one that masks individual observed values within each function, and the other that masks entire functions, both with masking probability of 0.5. The corresponding denoising losses for these two masking strategies (see (10)) are summed to form the final loss used for training the model.
dictive performance across all the tasks. In Darcy Flow (D-F), Convection Diffusion (C-D) and Diffusion Reaction (D-R), ACM-FD outperforms FNO in all tasks except for the fi, in to f2 mapping. In the Torus Fluid (T-F) system, while ACM-FD ranks second, its performance remains close to that of FNO. Furthermore, the performance of ACM-FD is consistently better than Simformer and other neural oper- ators, including GNOT, DON and PODDON (see Table in the Appendix). Despite being specifically optimized for each task, those neural operators are constantly worse than our method, underscoring the advantage of training a single, versatile model. The observed improvement of Simformer
over GNOT in many cases (both are based on transformer architectures) also implied the effectiveness of this strategy.
5.2 Generation Performance
We utilized 1, 1,000 instances for training, 100 instances for validation, and 200 instances for testing. Hyperparameter tuning was performed using the validation set. The details are provided in Appendix Section B. Following the evalu- ation procedure in (Lu et al., 2022), for each method, we selected the optimal hyperparameters, and then conducted stochastic training for five times, reporting the average rel- ative L2 test error along with the standard deviation. The results are presented in Table 1. Due to the space limit, the comparison results with PODDON are provided in Ap- pendix Table 5.
We then evaluated the generation performance of ACM-FD by using it to generate instances of all relevant functions for the Darcy Flow, Convection Diffusion and Torus Fluid systems. We compared ACM-FD with two methods. The first one multi-function diffusion (MFD) solely for gen- erating all the functions. We used the same architecture as ACM-FD but trained the model unconditionally, in accor- dance with the standard DDPM framework (see (9)). The second method is 3-VAE (Higgins et al., 2022). a widely used Variational Auto-Encoder (VAE), where / controls the regularization strength from the prior. The encoder network was built using a series of convolutional layers, while the
As we can see, ACM-FD consistently achieves top-tier pre-
7

---

Arbitrarily-Condtionn Multi-Functional Diffusion for Multi-Physscs Emulation
decoder network was constructed as the transpose of these layers. For training and hyperparameter tuning of 3-VAE, we used the same training and validation sets those used by ACM-FD in Section 5.1. The details are provided in Appendix Section B.
Table 2. Equation relative L2 error and diversity of generated data for the whole system. MRPD is short for Mean Relative Pairwise Distance.
System
Task(s)
ACM-FD
MFD
3-VAE
We used two metrics to evaluate the quality of the generated data. The first is the equation error, which measures how well the generated data adheres to the governing equations of the system. To calculate this error, for each group of sampled source function, parameter function, and/or initial conditions, we ran the numerical solver- - the same one used to generate the training data - to solve the govern- ing cquation(s), and then computed the relative L2 error between the generated solution and the numerical solution. The second metric is the Seean-RRlative-Paivvee Pairiise-Dsstance (MRPD), which measures the diversity of the generated data (Yuan & Kitani, 2020; Barquero et al., 2023; Tian et al., 2024). It is worth noting that since there is no large pre- trained inception network tailored to encompass all types of functions across numerous multi-physics systems, the popular FID score (Heusel et al., 2017) is not applicable to our evaluation.
D-F
Equation Error
0.0576
0.0584
0.265
MRPD
1.15
0.980
0.932
Equation Error
0.114
0.127
0.282
C-D
MRPD
1.00
0.971
0.879
Equation Error
0.0273
0.0234
0.737
T-F
MRPD
0.8042
0.9537
0.524
Table 3. Relative Lu error for function completion. Each function is sampled in one half of the domain, while the other half is com- pleted using different methods.
Dataset
Task(s)
ACM-FD
MFD--npaint Interp
1.21e-02
7.94e-02
1.04e-01
D-F
f
1.23e-02
6.41e-02
6.98e-01
1.09e-02
2.71c-02
8.07c-01
V
1.87e-02
4.71e-01
8.30e-01
C-D
3,39c-02
3.22e-01
6.49e-01
1.45c-02
3.47e-02
8,97e-01
For each system, we generated 1,000 sets of the relevant functions. The average equation error and MRPD are pre- sented in Table 2. As shown, the functions generated by ACM-FD and MFD consistently exhibit relatively small equation errors, indicating strong alignment with the phys- ical laws governing each system. In contrast, the equa- tion errors for functions generated by 8-VAE are signifi- cantly larger, often by an order of magnitude. Furthermore, the data generated by ACM-FD and MFD demonstrates higher MRPD compared to B- B-V,, indicating better diver- sity among the generated function instances. For the func- tions generated by ACM-FD and MFD, both the equation error and MRPD are comparable. Specifically, in Darcy Flow (D-F) and Convection Diffusion (C-D), ACM-FD achieves smaller equation errors and higher MRPD, while in Torus Fluid (T-F), the metrics for ACM-FD are slightly worse. These results suggest that ACM-FD, despite being trained for a variety of prediction and conditional generation tasks, still achieves comparable unconditional generation performance to the diffusion model trained purely for un- conditional generation. Overall, these findings indicate that ACM-FD capable of producing multi-physies data that is not only reliable but also diverse. In Appendix Section C, we provide examples of the functions generated by each method, along with detailed discussion and analysis.
(C-D) systems. For each function, we provided samples from half of the domain and used ACM-FD to predict the function values in the other half. To benchmark perfor- mance, we adapted our multi-functional diffusion (MFD) framework to the DDPM inpainting approach proposed in (Lugmayr al., 2022), which we refer to MFD-Inpaint. This method trains MFD unconditionally as in Section 5.2. During the completion process, the observed function values are perturbed using the forward process, and these perturbed values, along with noise for the unobserved function values, are passed into the denoising network. Additionally, we compared ACM-FD with the interpolation strategy, which commonly used scientific and engineering domains. We employed scipy: - i inte interpol ate gr ddat. t for inter- polation. We denote this method as Interp. The relative L2 error for each method is presented in Table 3. As shown, ACM-FD consistently achieves much lower errors compared to both MFD-Inpaint and Interp. The superior performance of ACM-FD over MFD-Inpaint might be attributed to its explicitly conditional training strategy, which allows ACM- FD leverage conditioned information more effectively during completion. It is also noteworthy that both ACM-FD and MFD-Inpaint substantially outperform classical inter- polation, demonstrating that diffusion- modeling is far more effective for inferring unknown function values. Completion examples for each method are visualized in Appendix Fig. 6.
5.3 Completion Performance
Third, we evaluated ACM-FD on the task of completing functions in unobserved domains. Specifically, we tested ACM-FD on Darcy Flow (D-F) and Convection Diffusion
8

---

Arbitraril--Condttonee Multi-Functional Diffusion for Multi-Physies Emulation
Ground Truth
Mean Prediction.
Abselute Errar Standard Deviation
Table Empirical Coverage Probability (ECP) under confidence
levels {0.9, 0.95, 0.99). Best results per task are in boldface.
Dataset
Task
Method
0.9
0.95
0.99
ACM-FD
0.833
0.880
0,921
8, u to
Simformer
0.736
0.814
0.871
(a) D-F: u to
ACM-FD
0.766
0.842
0.913
U, to
Simformer
0.683
0.767
0.879
ACMFFD
0.939
0.968
0.990
C-D
v. 8 to
Simformer
0.695
0.771
0.858
ACM-FD
0.821
0.870
0,922
to
Simformer
0.775
0.850
0.912
ACM-FD
0.920
0.949
0.972
(b) D-F: to f.
to
Simformer
0.716
0.773
0.823
ACM-FD
0.947
0.974
0.991
a, u to
Simformer
0.829
0.895
0.950
ACM-FD
0.985
0.994
0.998
D-F
a. f to ttoof
Simformer
0.922
0.955
0.998
ACM-FD
0.867
0.909
0.952
(c) D-F: 0, f to
Simformer
0,918
0.953
0.980
5.4 Uncertainty Quantification
(d) C-D: to v.
Fourth, assessed the quality of uncertainty calibra- tion provided by ACM-FD. To this end, we examined the empirical coverage probability (Dodge, 2003): ECP = N E.E1 T(y; E C.), where y; is the ground-truth func- tion value, and C. is the a confidence interval derived from 100 predictive samples generated by our method. We var- ied from (90%, 95%, 99%], J, and examined our method in eight prediction tasks across the Convection-Diffuse (C-D) and Darcy Flow (D-F) systems. We compared against Simformer. Note that all the other competing methods are deterministic and therefore unable to perform uncertainty quantification. As shown in Table 4, in most cases our method achieves coverage much closer a, showing supe- rior quality in the estimated confidence intervals.
(c) C-D: to
Figure 2. Visualizations of ACM-FD predictions.
6 Conclusion
We have presented ACM-FD, novel probabilistic genera- tive model designed for multi-physics emulation. ACM-FD can perform aubitrarily-oondiioneed multi-function sampling and has demonstrated potential in several classical systems. In future work, we plan to extend our investigations into more complex multi-physics systems and explore alternative architecture designs further extend its capabilities.
Additionally, visualized prediction examples along with their uncertainties, measured by predictive standard devia- tion (std). As shown in Figure 2, more accurate predictions tend to have lower std, i.e., low uncertainty, while regions with larger prediction errors correspond to higher std providing further qualitative evidence that our uncertainty estimates are well-aligned with prediction quality.
A Acknowledgements
SZ. DL and ZX acknowledge the support of MURI AFOSR grant FA9550-2001100358, NSF CAREER Award IIS-2046295, NSF OAC-2311685, and Margolis Founda- tion. AN acknowledges the support of the grant AFOSR FA9550-23-1-074 This work has also leveraged NCSA Delta GPU cluster from NCF Access program under Project CIS240255.
5.5
Ablation Studies
Finally, conducted ablation studies to assess the contribu- tion of the key components in ACM-FD, including rundom masking strategy. masking probability p. and Kronecker product-based computation. The results and analysis, as presented in Appendix Section D, confirm the effectiveness of each component.
Impact Statement
This paper presents work whose goal to advance machine learning for multi-physies emulation. There are many po-

---

Arbitrarity-Cnditioncd Multi-Functional Diffusion for Multi-Physscs Emulation
tential societal consequences of our work, none of which we feel must be specifically highlighted here.
Kennedy, M. C. and O'Hagan, A. Predicting the output from complex computer code when fast approximations are available. Biometrika, 87(1):1-13, 2000.
References
Kerrigan, G., Ley, J., and Smyth, P. Diffusion gener- ative models in infinite dimensions. In International Conference on Artificial Intelligence and Statistics, pp. 9538 9563. PMLR, 2023.
Barquero, G., Escalera, S., and Palmero, C. Belfusion: Latent diffusion for behavior-driven human motion pre- diction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2317-2327, 2023.
Kerrigan, G., Migliorini, G., and Smyth, P. Functional flow matching. In International Conference Artificial Intelligence and Statistics, pp. 3934-3942, PMLR, 2024.
Batlle, P., Darey, M., Hosseini, B., and Owhadi, H. Kernel methods are competitive for operator learning. arXiv preprint arXiv:2304. 13202, 2023.
Klein, L.,, Krämer, A., and Noé, Equivariant flow matching. Advances in Neural Information Processing Systems, 36, 2023.
Chung, H., Kim, J., McCann, M. T., Klasky, M. L., and Ye, J. C. Diffusion posterior sampling for general noisy inverse problems. In 1 1th International Conference on Learning Representations (ICLR), 2023.
Kolda, T. G. Multilinear operators for higher-order decompositions, volume 2. United States. Department of Energy, 2006.
Dodge, Y. The Oxford dictionary of statistical terms. Ox- ford University Press, USA, 2003.
Kovachki, N. B., Li, Z., Liu, B., Azizzadenesheli, K., Bhat- tacharya, K., Stuart, A. M., and Anandkumar, A. Neural operator: Learning maps between function spaces with applications to pdes. J. Mach. Learn. Res., 24(89):1-97, 2023.
Franzese, G., Corallo, G., Rossi, S., Heinonen, M., Filip- pone, M., and Michiardi, P. Continuous-timm functional diffusion processes. Advances in Neural Information Processing Systems, 36, 2024.
Li, S., Wang, Z., Kirby, R. M., and Zhe, S. Deep multi-fidelity active learning of high-dimensional out- puts. Proceedings of the Twenty-Fifth International Conference on Artificial Intelligence and Statistics, 2022.
Gloeckler, M., Deistler, M., Weilbach, C., Wood, F., and Macke, J. H. All-in-one simulation-based inference. arXiv preprint arXiv:2404.09636, 2024.
Gupta, G., Xiao, X., and Bogdan, P. Multiwavelet-based operator learning for differential equations. Advances in neural information processing systems, 34:24048-2406,, 2021.
Li, S., Yu, X., Xing, W., Kirby, R., Narayan, A., and Zhe, S. Multi-resolution active learning of Fourier neu- ral operators. In International Conference on Artificial Intelligence and Statistics, pp. 2440-2448, PMLR, 2024.
Hao, Z., Wang, Z., Su, H., Ying, C., Dong, Y., Liu, S., Cheng, Z., Song, J., and Zhu, J. Gnot: A gen- eral neural operator transformer for operator learning. In International Conference on Machine Learning, pp. 12556-12566 PMLR, 2023.
Li, Z., Kovachki, N., Azizzadeneshelii K., Liu, B., Bhat- tacharya, K., Stuart, A., and Anandkumar, A. Neural operator: Graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03485, 2020a.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. GANs trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Li, Z., Kovachki, N. B., Azizzadeneshhlli K., Bhattacharya, K., Stuart, A., Anandkumar, A., et al. Fourier neural op- erator for parametric partial differential equations. In International Conference on Learning Representations, 2020b.
Higgins, I., Matthey, I., Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., and Lerchner, A. beta- VAE: Learning basic visual concepts with constrained variational framework. In International Conference on Learning Representations, 2022.
Lim, H., Kovachki, N. B., Baptista, R., Beckham, C., Azizzadenesheli, K., Kossaifi, J., Voleti, V., Song, J., Kreis, K., Kautz, J.. al. Score-based diffusion models in function space. arXiv preprint arXiv:2302.074000 2023.
Ho, J., Jain, A., and Abbeel, Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-683 66511, 2020.
Lipman, Y., Chen, R. T., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
10

---

Arbitrarily Multi-Functional Diffusion for Multi-Physies Emulation
Long, D., Mrvaljevic, N., Zhe, S., and Hosseini, B. A kernel approach for PDE discovery and operator learning. arXiv preprint arXiv: 10.088 2022.
Tian, S., Zheng, M., and Liang. X. Transfusion: A prac- tical and effective transformer-basedd diffusion model for 3D human motion prediction. IEEE Robotics and Automation Letters, 2024.
Lowery, M., Turnage, J., Morrow Z., Jakeman, J. D., Narayan, A., Zhe, S., and Shankar, V. Kernel neu- ral operators (KNOs) for scalable, memory-efficiet, geometrically-fexibbee operator learning. arXiv preprint arXiv:2407.00809, 2024.
Tran, A., Mathews, A.. Xie, L., and Ong, C. S. Factorized Fourier neural operators. In The Eleventh International Conference on Learning Representations, 2023.
Wainwright, M. J., Jordan, M. I., et al. Graphical models, ex- ponential families, and variational inference. Foundations and Trends@ in Machine Learning. 1(1-2):1-055 2008.
Lu, L., Jin, P., Pang, G., Zhang, Z.., and Karniadakis, G. E. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218
2021.
Wang. Y., Yu, J., and Zhang, age restoration using denoising diffusion null-space model. In The Eleventh International Conference on Learning Representations, 2023a. URL https: openreview net Forum? weweooorunnidiiieeeeeeeeee wiiiiiiieggettT
Zero-shot im-
Lu, L., Meng, X., Cai, S., Mao, Z., Goswami. S., Zhang, Z., and Karniadakis, G. E. A comprehensive and fair comparison of two neural operators (with practical exten- sions) based on fair data. Computer Methods in Applied Mechanics and Engineering, 393:1147777 2022.
Wang. Z., Li, S., Fang, S., and Zhe, S. Diffusion-generatiin fultitfidelity learning for physical simulation. arXiv preprint arXiv:2311.05606. 2023b.
Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., and Van Gool, L. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11461-11471, 2022.
Wen, G., Li, Z.., Arizzadenesheli, K., Anandkumar, A., and Benson, S. M. U-FNO-an enhanced Fourier neural operator-basedt deep-learning model for multiphase flow. Advances in Water Resources, 163:104180, 2022.
Mitchell, A. R. and Griffiths, D. F. The finite difference method in partial differential equations. Number BOOK. John Wiley, 1980.
Yuan, Y. and Kitani, K. Dlow: Diversifying latent flows for diverse human motion prediction. In Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK, August 23 23-28, 2020, Proceedings, Part IX 16, pp. 346-364. Springer, 2020.
Rasmussen, C. E. and Williams, C. K. I. Gaussian Processes for Machine Learning. MIT Press, 2006.
Razavi, S., Tolson, B. A., and Burn, D. H. Review of surrogate modeling in water resources. Water Resources Research, 48(7), 2012.
Zhang. B. and Wonka, P. Functional diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4723-4732, 2024.
Song. J., Meng, C., and Ermon, S. Denoising diffusion implicit models. In International Conference on Learning Representations, 202la.
Zienkiewicz, O. C., Taylor, R. L., Zienkiewicz, O. C., and Taylor, R. L. The finite element method, volume 36. McGraw-hil London, 1977.
Song. J., Vahdat, A., Mardani, M., and Kautz, J. Pseudoinvers--guided diffusion models for inverse problems. In International Conference on Learning Representations, 2023.
Song. Y.. Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. In International Conference Learning Representations, 2021b. /ttttppe///ererreeieee..... formmmmm? id=pxTIG12RRHS,
Takamoto, M., Praditia, T., Leiteritz, R., MacKinlay, D., Alesiani, F., Pflüger, D., and Niepert, M. PDEBench: An extensive benchmark for scientific machine learning. Advances in Neural Information Processing Systems, 35: 1596-1611, 2022.

---

Arbitrarily-Condttonee Multi-Functional Diffusion for Multi-Physies Emulation
Appendix
A Mulit-Physics Systems Details
A.1 Darcy Flow
We considered single-phase 2D Darey flow system, which is governed by the following PDE,
-V -V- (a(x)Vu(x)) f(x x E (0, u(x) = 0, x E 0(0, 1)²;
(14)
where ax) is the permeability field, u(x) is the fluid pressure, and f(x) is the external source, representing sources or sinks of the fluid. To obtain one instance of the triplet (a, u.), we sampled from a Gauss random field: ~
(O, (-A A + - and a from the exponential of a Gaussian random field: a = exp(g) where ~ (O, (-A 16)-
then ran second-order finite-difference solver to obtain u at a 256 256 mesh. We extracted each function instance from 64 x 64 sub-mesh.
A.2 Convection Diffusion
We considered a ID convection-diflusionn system, governed by the following PDE,
Öu(x,t) t) Ot
V (v(2, uV + s2x
(15)
where (x, E 1-1,1] x [0, w(x, 0) 0. D is the diffusion coefficient and was set to 0.01, v(z, t) is the convection velocity, describing how fast the substance is transported due to the flow, s(x; t) is the source term, representing the external force, and u(x; 1) is the quantity of interest, such as the temperature, concentration and density. We employed a parametric form for v and a: v(x; t) = Σn= a + agt, and s(x, t) = a exp( -3 + t)?) + 7 cos(1) л(x 0.1t)), where all on a, 3, y, are sampled from Uniform(-1, 1). We used the MATLAB PDE solver pdepe' to obtain the solution for u. The spatial-temporal domain was discretized into 512 x 256 mesh. We extracted the function instances of interest from a 64 64 equally-spaced sub-mesh.
A.3 Diffusion Reaction
We used the 2D diffusion-reaction dataset provided from PDEBench (Takamoto et al., 2022). The dataset was simulated from a 2D diffusion- system, which involves an activator function Un (f, x; y) and an inhibitor function o2 x. y). These two functions are non-linearly coupled. The governing equation is given as follows.
Ov1
8201
8e
D
+ D
+
1/ K U2
01
O.2-
0y²
Ou2
82002
Øt
D2
Or.2
+ D2
0y²
+
122
(16)
where 2, (-1,1), (0,5), h 0.005, and the diffusion coefficients Dj 0.001 and D3 0.005. The initial condition is generated from the standard Gaussian distribution. The numerical simulation process detailed in (Takamoto et al., 2022). We are interested in four functions: fi vi (2.5, x, y). f2 (2 x, v). Ui = 0i (5.0, ar, y) and u2 (5.0. x. y). We extracted each function instance from a 64 x 64 sub-mesh from the numerical solutions.
A.4 Torus Fluid
We considered vicous, incompressive fluid on the unit torus. The governing equation in vorticity form is given by
Ow((x t) Øt + uuu Vi((x t) = vV f(x), w(x 0) = wo (x),
(17)
(18)
html
12

---

Arbitrarily Multi-Functional Diffusion for Multi-Physsis Emulation
where [0, 11.. E 10) 0.001, is the velocity field that V O, w<(x () is the vorticity function, and fxx is the source function that represents the external forces. We are interested in seven spatial functions: the initial
condition wo. the source function f. and the vortocity fields at time steps (=1 a1 + 31)), sin (a2 + {2))] A w1, and Wg. We employed a parametric form for wy and f: "u(x) = [sin π
2, 4, 6, 8, 10, denoted as W1, W2. W3.
[cos(a) (2g + 91)), (05(0227(222 32))]" where an, 02 Uniform(0.5, 1), $1, B2 Uniform(O, 1), and each [A] Uniform(-14 (-1, 1), and f (2) = 0.1 (z) + 22 + c)) + (2 + 22 + d)] where a, b Uniform(0, 2) and d ~ Uniform(O, 0.5). We used the inite-difference solver as provided in (Li et al., 2020b) to obtain the solution of {W1, W2, W3, Wig woy. The spatial domain was discretized to a 128 x 128 mesh. Again, to prepare the training and test datasets, we extracted the function instances from a 64 x 64 sub-mesh.
B Hyperparameter Selection
In the experiment, we used the validation dataset to determine the optimal hyperparameters for each method. The set of the hyperparameters and their respective ranges are listed as follows.
FNO: the hyperparameters include the number of modes, which varies from {12, 16, 18, 20, 24), the number of channels for channel lifting, which varies from (64, 128, 256), and the number of Fourier layers, which varies from (2, 3, 5)..
We used GELU activation, the default choice in the original FNO library?.
the hyperparameters include the number of convolution layers in the branch net, which varies from (3, 7), the kernel size in the convolution layer, which varies from (3, 5, 7J, the number of MLP layers in the truck net, which varies from (3, 4, 5), the output dimension of the branch net and trunk net, which varies from [64, 128, 256), and the activation, which varies from (ReLU, Tanh).
DON:
PODDON:
the hyperparameters include the number of bases, varying from (128, 256, 512), the number of convolution
layers in the branch net, varying from (3, 5, 71, the kernel size, varying from [3, 5, 7), the output dimension of the branch net and trunk net, varying from [64, 128, 256), and the activation, varying from ( (ReL.U, Tanh).
the hyperparameters include the number of attention layers, varying from [3. 4, 5), the dimensions of the embeddings, varying from [64, 128, 256], and the inclusion of mixture-of-exper--baee gating. specified as either [yes, no). We used GeLU activation, the default choice of the original library?.
GNOT:
Simformer (Gloeckler et al., 2024): the original Simformer was developed and tested on a small number of tokens. To handle a large number of sampling locations, which in our experiments amounts to 64 x 64 = 4, 096, we employed the linear attention mechanism as used in GNOT. The number of attention layers was varied from (3,4, 5). The dimensions of the embeddings were selected from (128, 256, 512). The activation was selected from (GELU, ReL.U, Tanh).
ACM-FD: the hyperparameters include the number of modes, which varies from [12, 16, 18, 20, 24), the number of channels for channel lifting, which varies from (64, 128, 256), the number of Fourier layers from, which varies from (3, 4, 5), the length-scale of the SE kemel, which varies from (1e-2, 5e-3, 1e-3, 5e-4, le-4). We used GELU activation.
3-VAE: the hyperparameters include 3, varying from [lel, le-1, 1e-3, le-4, le-5, le-6), the rank, varying from (16, 32, 64, 128, 256). and the number of convolution layers in the decoder and encoder networks, varying from from {1, 2, 3, 4). We employed the GELU activation and fixed the kernel size be 3.
For FNO, DON, PODDON and GNOT, each task underwent an independent hyperparameter tuning process to identify the optimal hyperparameters specific to that task. In other words, each model was retrained from scratch for each individual task. For example, solving ten tasks would result in ten distinct FNO models. In contrast, for Simformer and ACM-FD, the model was trained only once, where the validation enror is defined as the summation of the relative L2 error across all the tasks. The same model is used for inference on every task. Note that, 3- is only trained for data generation (not for any prediction ask). The tuning of 3-VAE is guided by the reconstruction error on the validation dataset.
Shttps://ge thub. com neura htmm//eurrlooeera.oom operaterrroooooo hwtp//nuuurarrara
Pttpps: ptttps://ghe //gi thub. com commHao@hoongkal /
13

---

Arbitrarily Multi-Functionll Diffusion for Multi-Physics
Emulation
9
u-sol
0
u-sol
(a) Darcy Flow
(b) Convection Diffusion
Wp
Wi
Wi-sol
W3
W3-S0l
Wg
Wg-SSl
(c) Torus Fluid
Figure Function instances generated by ACM-FD (top row), by MFD (middle row), and by B-VAE (third row). "-sol" means numerical solution provided by the numerical solvers given the other functions.
C Multi-physics System Generation
We investigated the functions generated by each method. Figure 3 illustrates a set of functions randomly generated by ACM-FD, MFD, and 3-VAE for Darcy Flow, Convection Diffusion and Torus Fluid systems. More examples are provided Figure 4 and 5. As we can see, the generated solutions from both ACM-FD and MFD are highly consistent with numerical solutions, accurately capturing both the global structures and finer details. In contrast, the solutions produced by 3-VAE often fail capture local details (see Fig. 3a for and Fig. for wi) or significantly deviate from the numerical solution in terms of overall shape (see Fig. 3b for u and Fig. 3e for wz). In addition, other functions generated by 3- B-VAE, such as (a,g) g) for Darcy Flow and (), for Convection Diffusion, appear quite rough and do not align well with the corresponding function families (see Section A). This might be due to that 3- 3-VAE is incapable of capturing the correlations between function values across different input locations. contrast, ACM-FD introduces GP noise functions and performs diffusion and denoising within the functional space. As a result, ACM-FD is flexible enough to effectively capture the diverse correlations between function values. Overall, these results confirm the advantages of our method in multi multi-physies
14

---

Arbitrarily-onndtiinne Multi-Functionll Diffusion for Multi-Physics Emulation
a
tt-80l
U
U-So]
(a) Darcy Flow
(b) Convection Diffusion
Wi
W1
W--sol
W3
W3-sol
W/
W3-sol
(c) Torus Fluid
Figure More generated function instances by ACM-FD, "-sol" means the numerical solution provided by the numerical solvers given the other functions.
15

---

Arbitrarity-onndtionne Multi-Functionll Diffusion for Multi-Physies Emulation
a
u-sol
U
U-sol
(a) Darcy Flow
(b) Convection Diffusion
W0
Wi-sol
W3
W3-
W!
Wg-S0l
(c) Torus Fluid
Figure 5. More generated function instances by MFD. "-sol" means the numerical solution provided by the numerical solvers given the other generated functions.
16

---

Arbitrarily-Condttonee Multi-Functional Diffusion for Multi-Physses Emulation
data generation.
Table 5. Relative Le error of ACM-FD vs. PODDON for various prodiction tasks. The results were averaged over five runs. D-F, C-D and T-F are short for Dancy Flow, Convection Diffusion, and Torus Fluid Systems.
Dataset
Task(s)
ACM-FD
PODDON
f. u to a, u a, f
1.32c-02 (2, (8)-8)
1.56e-02 (1.44e-04)
1.59c-02
(1.59c-04)
4.63e-02 (1.21e-03)
D-F
1.75e-02 (4.16e-04)
6.80e-02 (2.16e-04)
u u a
3.91e-02 (7.08e-04)
4.09c-02 (4.18c-04)
to f
3,98e-02 (6,45e-04)
6.37e-02 (1.03e-03)
u
2.17e-02 (4.53e-04)
4.62e-02 (2.00e-04)
to
5.45e-02 (1.40e-03)
7.57e-02 (4.09e-04)
C-D
t u
1.60e-02 (2.15e-04)
1.72e-01 (1.17e-03)
2.66e-02 (3,08e-04)
5.38e-02 (5.29e-04)
t
6.06e-02 (2.54e-04)
1.03e-01 (1.29c-03)
u,
1.44e-02 (8.96e-04)
3.85e-01
(2.00e-04)
f1, u1 to u2
1.59c-02 (3.68e-04)
2.67e-01 (4.09e-04)
D-R
f2, to fi
4.10e-02 (8.93e-04)
5.09c-01
(1.17c-03)
f2,
5.86e-02
(3,43e-04)
3.70c-01 (5.29c-04)
wo, wa to wn
2.73e-02
(4.78c-03)
6.06e-02
(2.91e-04)
Wo, Ws to W2
2.43e-02
(1.60e-03)
7.71c-02 (1.63c-04)
Wo, Ws to wy
2.43e-02
(3.17e-03)
7.38c-02 (2.92c-04)
Wo, Ws W4 wo, ws f
1.68e-02
(1.81e-03)
5.38c-02 (2.18c-04)
1.63e-02
(1.49c-03)
4.94e-02 (9.02e-04)
T-F
wo. f to wi
3.10c-02
(4.08c-03)
5.54e-02 (6.45e-04)
wo. f to u2
3,28c-02 (4.79c-03)
7.40e-02 (2.01e-04)
Wo. f to
3,49e-02 (2.38e-03)
8, 0.60e-02 (5.16e-04)
Wo. f to 3.34e-02
(3.87e-03)
9.74e-02 (4.21e-04)
Wo. f to Ws
3.26e-02 (2.13e-03)
1.16e-01 (8.07e-04)
D Ablation Studies
In this section, we conducted ablations studies to assess the effect of critical components of our method.
Random Masking. First, we evaluated the importance of our random masking strategy in model training. To this end, compared ACM-FD with MF1) the variant that trained without random masks. That is, the training is fully unconditional. Table 6 shows the relative L2 error in D-F and C-D systems. As see, across various prediction tasks, our model trained with random masks consistently outperforms the variant without them, achieving substantial error reductions ranging from 40.9% to 96.2%.
Choice of Masking Probability. Next, we investigated the impact of the masking probability p. In the main paper, all results were obtained using neutral setting of p 0.5, which yields the highest masking variance. Here, varied p across (0.2, 0.4, 0.6, 0.8) to evaluate the prediction accuracy of our method on the D-F and C-D systems. As shown in Table 7, the performance at = 0.4 and 0.6 remains comparable to that at 0.5, indicating degree of robustness to the choice of p. However, more extreme values, such as 0.2 or 0.8, lead to a noticeable drop in performance.
Kronecker Product based Computation. Third, we evaluated the impact of incorporating the Kronecker product in our method and compared both training and inference time with other approaches. All runtime experiments were conducted on a Linux cluster node equipped with an NVIDIA A GPU (40GB memory).
To ensure a fair and comprehensive comparison, we first measured the per-epoch training time. For neural operator haselines, we recorded the total per-epoch time across all the tasks. The results, as summarized in Table 8, show that leveraging Kronecker product properties greatly improves the training efficiency of our method. Our per-epoch training time substantially lower than that of all competing methods except DON.
However, diffusion- based models typically require far more training epochs than deterministic neural operators. For example, FNO and GNOT converge within K epochs in all the settings, while our method generally requires around
17

---

Arbitrarily Multi-Functioall Diffusion for Multi-Physies Emulation
Ground-Truth
ACM-FD
MFD-Inpaint
Interp
(a) Darcy Flow: examples of completing functions (the first row), (the second row) and u (the third row).
Ground-Truth
ACM-FD
MFD-Inpaint
Interp
(b) Convection Diffusion: examples completing functions (the first row), s (the second row), and u (the third row).
Figure 6. Examples of function completion. The functions are sampled from the left half of the domain, and the right half are completed by each method.
18

---

Arbitrarily-Condttoned Multi-Functional Diffusion for Multi-Physies Emulation
20K epochs. As result, despite the per-epoch efficiency, the total training time for method as well for Simfomer, another diffusion-based model remains higher overall.
Lastly, by using Kronecker product, our method achieves substantial acceleration in generation and/or prediction, with 7.4x speed-up C-D and 6.9x speed-up D-F. During inference, the computational cost is dominated by sampling noise functions, whereas during training, substantial portion of cost arises from gradient computation. Consequently, the runtime advantage of using the Kronecker product is even more pronounced during inference.
Table 6. Comparison MFD and ACM-FD in relative L2 errors.
Dataset
Task
MFD
ACM-FD
f, u a a, u to f
1.70e-1 (3.45e-3)
1.32e-2 (2.18e-4)
6.98e-2 (3.09e-3)
1.59e-2 (1.59e-4)
D-F
a. to u to
2.96e-2 (1.16e-3)
1.75e-2 (4.16e-4)
1.70c-1 (3.56c-3)
3.91e-2 (7.08e-4)
u
1.05e-1 1 (4.30e-3)
3.98e-2 (6.45e-4)
8,
5,47e-1 (3.56e-2)
2.17e-2
(4.53e-4)
0, s
3,95c- (4.01c-2)
5.45e-2 (1.40e-3)
C-D
U, to u
3.68e-2 (1.65e-3)
1.60e-2
(2.15e-4)
u tt
6.94c-1 (3.64e-2)
2.66e-2 (3.08e-4)
u 10
9.23e-1 (3.64e-2)
6.06c-2 (2.54c-4)
Table Relative L2 error across different masking probabilities
Dataset
Task
p-0.2
0.4
0.5
0.6
0.8
f, u to u to f a, f to 2.50e-2
2.16e-2
1.60e-2
1.32e-2
1.34c-2
1.26c-2
1.85e-2
1.61e-2
1.59e-2
1.67e-2
1.70c-2
D-F
1.95e-2
1.75e-2
2.05e-2
2.00e-2
4.48e-2
4.14e-2
3.91c-2
3.93e-2
4.96e-2
to 4.26e-2
4.07e-2
3.98e-2
4.32e-2
5.02e-2
8, 18
3.24e-2
2.72c-2
2.17e-2
2.33e-2
2.91e-2
U, u to
7.11e-2
6.85e-2
5.45e-2
5.860-2
8.35e-2
C-D
8,8 to 1.81e-2
1.760-2
1.60e-2
1.77c-2
3.56c-2
to 2.91e-2
2.4le-2
2.66e-2
2.65e-2
5.15e-2
7.69e-2
5.62e-2
6.06e-2
6.91e-2
9.15e-2
19

---

Arbitrarily-onndtiinne Multi-Functionll Diffusion for Multi-Physies Emulation
Table 8. Comparison of training and inference time C-D and D-F systems. ACM-FD (w/o K) means running method without using Kronecker product propertics for computation.
(a) Training time per epoch (in seconds)
Dataset
ACM-FD
ACM-ED (w/o K)
Reduction
Simformer
FNO
GNOT
DON
C-D
3.09
5.5
43.8%
23.6
16.45
144
1.53
D-F
3.27
5.55
41.4%
23.5
15.6
148
1.48
(b) Total training time (in hours)
Dataset
ACM-FD
ACM-FD (w/o K)
Sinformer
FNO
GNOT
DON
C-D
17.2
>
45.9
4.57
40
4.25
D-F
18.2
>48
45.7
4.33
41.1
4.11
(c) Inference time per sample (in seconds)
Dataset
АCM-FD
ACM-FD (w/o K)
Reduction
Simformer
C-D
0.899
6.66 6.7
86.5%
7.39
D-F
0.975
85.4%
7.34
20


Reviews:
[{"review_id": "5REoDkahcA", "reviewer": "Reviewer_bmeB", "summary": "The paper introduces Arbitrarily-Conditioned Multi-Functional Diffusion (ACM-FD), a novel probabilistic surrogate model designed for multi-physics emulation. It aims to address limitations of traditional machine learning-based surrogate models, which are typically task-specific and lack uncertainty quantification. ACM-FD is based on the Denoising Diffusion Probabilistic Model (DDPM) but extends it to handle multiple functions in multi-physics systems with a unified framework. The key contributions are:\n\n1. Multi-Functional Diffusion Framework: The paper introduces a framework based on DDPM that uses Gaussian Processes (GPs) for noise modeling, enabling the generation of multiple functions in multi-physics systems.\n2. Innovative Denoising Loss: A random-mask-based, zero-regularized denoising loss is proposed to handle conditional generation tasks, improving network stability and task flexibility.\n3. Efficient Training and Sampling: The model reduces computational costs by leveraging a Kronecker product structure and tensor algebra to simplify GP covariance matrix operations.\n4. Experiments: ACM-FD reaches top-tier performance in 24 prediction tasks across four multi-physics systems, achieving high accuracy, adherence to governing equations, and diversity in generated data. It also excels in function completion compared to inpainting and interpolation methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the thoughtful and constructive comments.\n\n> no analysis of UQ performance.\n\nR1: Great suggestion. Here we add UQ performance evaluation & analysis results.\n\nFirst, to evaluate UQ quality, we computed the **emprical coverage probability** [1]: $CP = \\frac{1}{N} \\sum_{i=1}^N I(y_i \\in C_\\alpha)$ where $y_i$ is the ground-truth function value, and $C_\\alpha$ is the $\\alpha$ confidence interval derived from 100 predictive samples generated by our method. We varied $\\alpha$ from {90%, 95%, 99%}, and examined our method in eight prediction tasks across Convection-Diffusion (C-D) and Darcy Flow (D-F) systems. We compared against Simformer. Note that the other baselines are deterministic and unable to perform UQ.\n\n**C-D**\n| Task  | Method | $\\alpha$=0.9   | 0.95 |  0.99|\n|-|-|-|-|-|\n| $s,u$ to $v$ | Ours | **0.833**  | **0.88**  | **0.921**   |\n|  | Simformer | 0.736  | 0.814  | 0.871   |\n| $v,u$ to $s$ | Ours | **0.766**  | **0.842**  | **0.913**   |\n|  | Simformer | 0.683  | 0.767  | 0.879   |\n| $v,s$ to $u$ | Ours | **0.939**  | **0.968**  | **0.99**   |\n|  | Simformer | 0.695  | 0.771  | 0.858   |\n| $u$ to $v$ | Ours | **0.821**  | **0.87**  | **0.922**   |\n|  | Simformer | 0.775  | 0.85  | 0.912   |\n| $u$ to $s$ | Ours | **0.92**  | **0.949**  | **0.972**   |\n|  | Simformer | 0.716  | 0.773  | 0.823   |\n\n**D-F**\n| Task  | Method | $\\alpha$=0.9   | 0.95 |  0.99|\n|----------|-----------|--------------------|----------------------------|-----------|\n| $a, u$ to $f$ | Ours | **0.947**  | **0.974**  | **0.991**   |\n|  | Simformer | 0.829  | 0.895  | 0.95   |\n| $a, f$ to $u$ | Ours | **0.915**  | **0.949**  | **0.998**   |\n|  | Simformer | 0.922  | 0.955  | **0.998**   |\n| $u$ to $f$ | Ours | 0.867  | 0.909  | 0.952   |\n|  | Simformer | **0.918**  | **0.953**  | **0.98**   |\n\nIn most cases our method achieves coverage much *closer* to $\\alpha$, showing superior quality in the estimated confidence intervals.\n\nNext, we visualized prediction examples along with their uncertainties, measured by predictive standard deviation (std). As shown [here](https://github.com/wjnzgzjyb/Arbitrarily-Conditioned-Multi-Functional-Diffusion-for-Multi-Physics-Emulation/tree/main), more accurate predictions tend to have lower std, i.e., low uncertainty, while regions with larger prediction errors correspond to higher std — providing further qualitative evidence that our uncertainty estimates are well-aligned with prediction quality. We will include the results in our paper.\n\n[1] Dodge, Y. (Ed.). (2003). The Oxford dictionary of statistical terms. Oxford University Press.\n\n> lacks a training-time comparison against baseline models.\n\nR2: Excellent suggestion. We have added an ablation study to assess the impact of incorporating Kronecker product in our method, and compared both the training  and inference time with other methods. For a fair and comprehensive evaluation, we first examined the per-epoch training time. For neural operator baselines, we measured their *total* per-epoch time across all the tasks.\n\n**Training time (per-epoch) in seconds**\n|System|Ours w/ Kronekcer | Ours w/o Kronecker |Runtime Reduction| Simformer |  FNO | GNOT | DON|\n|-|-|-|-|-|-|-|-|\n| C-D | 3.09  |5.5 | 43.8%| 23.6 | 16.45 | 144 | 1.53|\n| D-F | 3.27 | 5.55 |41.4%  | 23.5  |  15.6  |148| 1.48|\n\n**Training time (total) in hours**\n| System  | Ours w/ Kronekcer  | Ours w/o Kronekcer | Simformer |  FNO | GNOT | DON|\n|-|-|-|-|-|-|-|\n|C-D| 17.2 | >48 | 45.9 |4.57| 40 | 4.25|\n|D-F| 18.2 | >48 | 45.7 |4.33| 41.1| 4.11|\n\nThe results show that leveraging Kronecker product properties largely improves the training efficiency of our method. Our per-epoch training time is much less than all the competing methods except DON, which achieves exceptionally fast training by using PCA bases as the trunk net. However, diffusion models typically require far more training epochs than deterministic neural operators.\nFor instance, FNO and GNOT converge within 1K epochs across all the cases, while our method typically needs around 20K epochs. Consequently, despite per-epoch efficiency gains, our method, as well as Simfomer --- another diffusion-based method --- still takes longer overall training time.\n\n**Inference time (per-sample) in seconds**\n| System  | Ours w/ Kronekcer | Ours w/o Kronecker   |Runtime Reduction | Simformer |  \n|-|-|-|-|-|\n|C-D|0.899|6.66|86.5%|7.39|\n|D-F|0.975|6.7|85.4%|7.34|\n\nLastly, by using Kronecker product, our method achieves substantial acceleration in generation, with a 6.7x speed-up. During inference, the computational cost is dominated by sampling noise functions, whereas during training, a substantial portion of cost also arises from gradient computation. Consequently, the runtime advantage of using the Kronecker product is even more pronounced during inference. We will include these results and discussions in the paper.", "claims_and_evidence": "Most of the claims made in this paper are well supported by clear evidence.\n\nHowever, the authors have highlighted the importance of uncertainty quantification (UQ) and the advantage of their proposed method over the baselines by diffusion model naturally supporting UQ; while there is no analysis presented for UQ capabilities (i.e. ECE scores / confidence intervals that match the stochasticity in the data if any). \n\nThe authors are also claiming that the proposed method is efficient to train with Kronecker product, it would be supportive to have a training time comparison table to demonstrate the advantage. Since ACM-FD only needs to be trained once for each dataset, it would be reasonable to compare the total time of training other baselines for all prediction tasks regarding a dataset with the time ACM-FD needs.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are suitable for the problem presented. However, the uncertainty quantification performance is not evaluated although demonstrated as an advantage over other baseline models in the paper.", "theoretical_claims": "In this work, the authors focus on the methodological framework with empirical validation.", "experimental_designs_or_analyses": "The authors compare ACM-FD against various baseline models, such as Fourier Neural Operator (FNO), Transformer-based Neural Operator (GNOT), DeepONet (DON), and POD-based DON (PODDON). These baselines are compared thoroughly across 4 Physical systems: Darcy Flow, Convection-Diffusion, Diffusion-Reaction, and Torus Fluid Systems; together with 4 prediction tasks for each system: forward prediction, inverse inference, joint simulation, and function completion. \n\nACM-FD outperforms other baselines in most of the tasks. But FNO outperforms ACM-FD in 8 out of 10 Torus Fluid Systems tasks, resembling potential improvements for ACM-FD.", "relation_to_broader_scientific_literature": "ACM-FD advances beyond the baseline methods by enabling a single model to perform forward prediction, inverse inference, joint simulation, and function completion, with built-in uncertainty quantification. Works like PODDON have sought to improve efficiency in neural operators but still fall short in handling diverse prediction tasks. The generalization capability to different sub-tasks from one unified training stage can have great impact in reducing physics emulation costs.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n\n1. Paper is well written and easy to follow.\n2. The random masking and function discretization together provides a simple but efficient way to learn the full functional space (from data) during training.\n3. The proposed method is novel in Multi-Physics Emulation field and eliminates the need to retrain models for diverse prediction tasks.\n4. Empirical results show strong performance improvements.\n\nWeaknesses:\n\n1. Although authors have demonstrated the natural uncertainty quantification advantage with DDPM compared to the baselines, there is no analysis of UQ performance.\n2. Efficiency in training the DDPM model is also presented as a contribution, which lacks a training-time comparison against baseline models. For details of suggestion please refer to \"Claims And Evidence\" section in the review.", "other_comments_or_suggestions": "It would be clearer to have another figure to demonstrate how the inputs and outputs of the target function are discretized on the mesh.", "questions_for_authors": "Please refer to weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hXcqFqpnFm", "reviewer": "Reviewer_BtjJ", "summary": "The paper presents Arbitrarily-Conditioned Multi-Functional Diffusion (ACM-FD), a novel probabilistic surrogate model for multi-physics emulation. The key contributions include:\n\n- A multi-functional diffusion framework based on DDPM that models noise as multiple Gaussian processes, enabling generation of multiple functions in multi-physics systems\n- An innovative denoising loss using random masks that handles all possible conditional parts within the system\n- An efficient training and sampling approach using Kronecker product structure in the GP covariance matrix\n- Comprehensive experiments across four multi-physics systems showing superior performance compared to state-of-the-art neural operators\n\nThe paper demonstrates that ACM-FD can handle various tasks including forward prediction, inverse problems, completion, and data simulation within a single framework, while providing uncertainty quantification.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the many constructive comments.\n\n> Consider adding ablation studies to demonstrate importance of each component (random masks, Kronecker structure)\n\nR1: Great suggestion. Actually, we have already conducted studies on **the effect of random masks**. In Section 5.3, we compared our method (ACM-FD) with a variant trained **without** random masks (denoted as MFD-Inpaint) on the function completion task. As shown in Table 3, incorporating random masks during training significantly reduces the relative $L_2$ error --- by as much as 58\\% to 96\\%.\n\nHere, we additionally include results for prediction tasks, presented below. Across various prediction settings, our model trained with random masks consistently outperforms the variant without them (denoted as MFD), achieving substantial error reductions ranging from 40.9% to 96.2%.\n\n**Relative $L_2$ error**\n|System | Task  | MFD | ACM-FD  |\n|-|-|-|-|\nDF|$f,u$ to $a$|1.70e-1 (3.45e-3)|**1.32e-2 (2.18e-4)**|\n||$a,u$ to $f$|6.98e-2 (3.09e-3)|**1.59e-2 (1.59e-4)**|\n||$a, f$ to $u$|2.96e-2 (1.16e-3)|**1.75e-2 (4.16e-04)**|\n||$u$ to $a$|1.70e-1 (3.56e-3)|**3.91e-2 (7.08e-04)**|\n||$u$ to $f$ |1.05e-1 (4.3e-3)|**3.98e-2 (6.45e-04)**|\nCD|$s,u$ to $v$|5.47e-1 (3.56e-2)|**2.17e-2 (4.53e-04)**|\n||$v,u$ to $s$|3.95e-1 (4.01e-2)|**5.45e-2 (1.40e-03)**|\n||$v,s$ to $u$|3.68e-2 (1.65e-3)|**1.60e-2 (2.15e-04)**|\n||$u$ to $v$|6.94e-1 (3.64e-2)|**2.66e-2 (3.08e-04)**|\n||$u$ to $s$|9.23e-1 (3.64e-2)|**6.06e-2 (2.54e-04)**|\n\nWe have also supplemented the ablation study on the use of the Kronecker product structure. Please **see R2 to Reviewer bme8** for details.\n\n> discuss [1,2]\n\nR2: Thank you for the excellent references. We will definitely cite and discuss them. A key distinction is that the masks used in [1,2] are **fixed** and derived from the sampling patterns inherent to the medical imaging data. As a result, these works focus on a **single** inverse task --- either recovering high-frequency regions [1] or unsampled measurements [2]. In contrast, our method **repeatedly** samples **random masks** during training, enabling it to jointly handle a wide variety of conditional sampling tasks, encompassing all kinds of forward prediction, inverse prediction, and completion tasks.\n\nAnother critical difference lies in the **functional space formulation** of our model: the noises used during both training and inference are stochastic functions sampled from Gaussian processes. Our method is designed to generate functions either unconditionally or conditioned on other function samples. This motivation leads to fundamentally different designs in model architecture, training loss, and computational techniques (e.g., the use of Kronecker product).\n\n> Add more details about the training process and hyperparameter selection\n\nR3: Thank you for the great suggestion. We actually have provided detailed information in **Appendix Section B**, including the set of hyperparameters, their ranges, implementation details and libraries used for each method, as well as the validation settings. Due to space constraints, only a subset of this information is included in the main paper, with a reference to the appendix (see Line 366). In response to your feedback, we will further expand and enrich Appendix Section B to improve clarity and completeness.\n\n> Add statistical significance tests for the performance comparisons\n\nR4: Thank you for the excellent suggestion. Based on a $z$-test conducted on the prediction errors with associated error bars, our approach outperforms the baseline methods in the vast majority of cases at the 95% significance level. We will include the results in our paper.\n\n>Include more details about the training data generation process\n\nR5: We actually have included these details in **Appendix Section A**, which provides the governing equations, numerical simulation library used, simulation procedures, and data collection process. We believe this information is sufficient to generate all the data for our experimental settings. Additionally, we will publicly release our experimental data.\n\n> Provide visualization of the generated functions and their uncertainty estimates\n\nR6: Thank you for the great suggestion. We have added the suggested visualizations and analysis --- **please see our response R1 to Reviewer bmeB for details**.\n\n> Lack of comparison with traditional numerical methods\n\nR7: In fact, we did compare with traditional numerical methods, as all our **test data** (as well as the training data) are generated using them. We consider the outputs of the traditional methods as the **gold standard**, and our goal is to train a surrogate model that closely approximates their results. The relative $L_2$ error reported in Table 1-3 quantifies the difference of our method's prediction **as compared to the traditional methods**.", "claims_and_evidence": "The paper's claims are generally well-supported by evidence, with a few areas that could benefit from additional clarification:\n\nStrong Claims with Clear Evidence:\n- The ability to handle multiple functions of interest \n- The computational efficiency gains through Kronecker product structure\n- The superior performance across 24 prediction tasks\n\nAreas Needing Additional Support:\n- The claim about \"substantially reducing the training and sampling costs\" would benefit from quantitative comparisons with baseline methods", "methods_and_evaluation_criteria": "Suggestions for Improvement:\n- Consider adding ablation studies to demonstrate the importance of each component (random masks, Kronecker structure)\n- There are also some diffusion model works [1,2] that add masks during the diffusion process, and the reviewers believe that this needs to be discussed. NOTE: these two works are also using diffusion models to inverse problem.\n- Add more details about the training process and hyperparameter selection\n\n[1] High-Frequency Space Diffusion Model for Accelerated MRI\n\n[2] Measurement-conditioned denoising diffusion probabilistic model for under-sampled medical image reconstruction", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The experimental design is comprehensive but could be enhanced:\n\nSuggestions for Improvement:\n- Add statistical significance tests for the performance comparisons\n- Include more details about the training data generation process\n- Provide visualization of the generated functions and their uncertainty estimates", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Weaknesses:\n- Limited theoretical analysis of the proposed method\n- Lack of comparison with traditional numerical methods\n- Need for more detailed ablation studies\n- Could benefit from more extensive visualization of results", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FW8ufYaFnD", "reviewer": "Reviewer_hz2Z", "summary": "The authors outline a generative modeling technique to produce functional solutions to multi-function problems, with an emphasis on emulating multi-function physical systems. This is accomplished by casting the typical DDPM framework into the functional space, where the noise is framed as a Gaussian process. Then, DDPM effectively samples at sampling points along the functions domain, and this is extended to multiple functions simultaneously. Key contributions also include a tailored loss function for arbitrary conditioning, including conditioning on entire functions or on subsets of particular functional domains. They also provide an efficient scheme for sampling from the GP. They compare to extensive state-of-the art baselines, including neural operators and a diffusion-based method, exhibiting superior performance. In an ablation-like study, they verify that the explicit training using the masked loss and 0-filling is beneficial.\n\n## update after rebuttal: I believe my original score holds, and that this work is a good contribution.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their insightful comments and valuable suggestions.\n\n> It would be interesting to see how uncertainty intervals or estimates obtained from this method compare to those obtained from other capable methods, such as simformer.\n\nR1: Thank you for the great suggestion — we completely agree. We have added the evaluation and analysis of uncertainty quantification. Please see our **response R1 to Reviewer bmeB** for details.\n\n>The most important work in this domain is perhaps [1], with others including [2,3]. These methods need not be compared with experimentally, but it would be interesting to discuss the advantages and differences between these approaches\n\nR2: Thank you very much for providing these excellent references! We will certainly cite and discuss them.\n\nThe key difference lies in the training and inference paradigms. The referenced works [1–3] focus on training an **unconditional diffusion model**, and only at inference time do they incorporate the likelihood of observations or measurements to guide sampling from the conditional distribution. The advantage of this approach is that it only requires a standard diffusion model --- potentially even a pre-trained one --- without the need to address conditional sampling tasks during training. Indeed, our baseline **MFD-Inpaint** (see Section 5.2) follows this paradigm.\n\nHowever, the referenced methods may have limitations when applied to our scenarios:[1] requires a closed-form expression of the likelihood or forward model $p(y|x_0)$, or at least an efficient way to compute its gradient (or sensitivity) --- which is often infeasible in multi-physics systems governed by complex differential equations. The work [2] further restricts the applicability by assuming that the forward model is linear. [3] is more flexible and can handle nonlinear forward models, but it requires a \"pseudo-inverse\" operator for the forward process, which might not be easy to obtain.\n\n Our approach takes a different route: during training, we **explicitly incorporate conditionality** by randomly sampling masks and training our diffusion model to handle a wide range of conditional sampling tasks (including the unconditional case). At inference time, we **do not incorporate any likelihood scores** --- instead, we directly sample from our learned denoising model. Empirically, we found that this conditional training strategy yields **large improvements over unconditional training approaches** like MFD-Inpaint across a variety of tasks (see Table 3 and our response R1 to Reviewer BtjJ).\n\n> The arbitrary choice of h being Bernoulli-distributed with p=0.5 is not justified... Have you experimented with varying p through training?\n\nR3: Thank you for the insightful question. When there is no prior knowledge about whether a function value should be conditioned on or be generated, we suggested using $p = 0.5$ to avoid bias --- giving each component an equal chance of being conditioned or sampled. This is actually a uniform prior. We do agree, the mean of the proportion of the masked components is 0.5; but the standard deviation is also 0.5 ($\\sqrt{p\\times(1-p)}=0.5)$, which is the maximum possible, indicating that **the actual masked proportion can vary widely**. In our experiments, we used $p = 0.5$ to maintain a neutral setting. However, our method places no restrictions on the choice of $p$; it can be adjusted based on domain knowledge or specific application needs.\n\nWe have now included results for alternative values of $p$: 0.2, 0.4, 0.6, and 0.8. As shown below, the performance for $p=0.4$ and $p=0.6$ is comparable to that of $p=0.5$, demonstrating a certain degree of robustness to the choice of $p$. However, when $p$ is set to more extreme values, such as 0.2 or 0.8, the performance degrades markedly.\n\nWe will add these results and discussions into our paper.\n\n**Relative $L_2$ error**\n|System|Task|$p=0.2$|0.4|0.5|0.6|0.8|\n|-|-|-|-|-|-|-|\nDF|$f,u$ to $a$|2.16e-2|1.6e-2|**1.32e-2**|1.34e-2|1.26e-2|\n||$a,u$ to $f$|1.85e-2|1.61e-2|**1.59e-2**|1.67e-2|1.7e-2|\n||$a, f$ to $u$|2.5e-2|1.95e-2|**1.75e-2**|2.05e-2|2e-2|\n||$u$ to $a$|4.48e-2|4.14e-2|**3.91e-2**|3.93e-2|4.96e-2|\n||$u$ to $f$|4.26e-2|4.07e-2|**3.98e-2**|4.32e-2|5.02e-2|\nCD|$s,u$ to $v$|3.24e-2|2.72e-2|**2.17e-2**|2.33e-2|2.91e-2|\n||$v,u$ to $s$|7.11e-2|6.85e-2|**5.45e-2**|5.86e-2|8.35e-2|\n||$v,s$ to $u$|1.81e-2|1.76e-2| **1.60e-2**|1.77e-2|3.56e-2|\n||$u$ to $v$|2.91e-2|**2.41e-2**| 2.66e-2|2.65e-2|5.15e-2|\n||$u$ to $s$|7.69e-2|**5.62e-2**| 6.06e-2|6.91e-2|9.15e-2|\n\n> “We used FNO to construct our denoising network” What does this mean? You share the same architecture as FNO?\n\nR4: Great question. Yes, we use the same architecture as FNO - specifically, a sequence of Fourier layers — to construct our denoising network. We will make this point clearer in the paper.", "claims_and_evidence": "Their main claim is the advantage and flexibility of the arbitrary conditioning in generative models when solving diverse inverse problems. Training to solve all functions simultaneously, the authors demonstrate that their method outperforms methods trained specifically for each task individually, validating that the multi-function estimation approach is advantageous.", "methods_and_evaluation_criteria": "The authors rigorously evaluate their methods using L2 error on the reconstruction of functional outputs. Additionally, they assess the ability of their method to align with true physical equations, which is a particularly important consideration and often overlooked. Similarly, they assess the diversity in their outputs (exhibiting competitive performance). Finally, they fully examine the conditional capability of their approach for full-function prediction (forward models and inverse problems) and interpolation within a given function’s domain.", "theoretical_claims": "The authors do not make theoretical claims.", "experimental_designs_or_analyses": "The experiments sufficiently assess the validity of the approach. Natural questions might include (1) are neural operators trained for specific tasks more effective, (2) are other diffusion-like approaches more effective, particularly those with different conditioning strategies, and (3) how effective is the uncertainty quantification enabled by the method? The authors clearly and effectively answer questions (1) and (2) in the experiments, evaluating on a wide array of settings with competitive baselines. However, despite mentioning uncertainty quantification earlier in the paper, it is not addressed in the experiments. It would be interesting to see how uncertainty intervals or estimates obtained from this method compare to those obtained from other capable methods, such as simformer.", "relation_to_broader_scientific_literature": "The authors position their work within the context of generative modeling via diffusion. Specifically, they connect to other diffusion-based methods for functional generation/generating the outputs to a function, making the distinction that their work focuses on the generation of multiple related functions simultaneously. They also connect to the neural operator literature, which aims to learn function-to-function mappings, e.g., corresponding to solutions of PDEs. However, by leveraging diffusion/generative modeling, their framework naturally incorporates uncertainty quantification in addition to prediction.", "essential_references_not_discussed": "While not necessary for comparison in experiments, it might be good to acknowledge other diffusion-based methods for solving inverse problems, as that is a major focus of this work. The most important work in this domain is perhaps [1], with others including [2,3]. These methods need not be compared with experimentally, but it would be interesting to discuss the advantages and differences between these approaches and the outlined approach in the context of multi-function generation and inverse problems.\n\n[1] Chung et al. \"Diffusion Posterior Sampling for General Noisy Inverse Problems.\" ICLR 2023.\n[2] Wang et al. \"Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model.\" ICLR 2023.\n[3] Song et al. \"Pseudoinverse-guided diffusion models for inverse problems.\" ICLR, 2023.", "other_strengths_and_weaknesses": "I think this a strong work with rigorous experimental comparisons. While I was not initially convinced in the main text by the utility of the masked loss function for training a diffusion model capable of flexible conditional generation, it is made evident in the experiments that this is indeed necessary, as it is demonstrated to be more effective than prior methods with trained conditioning and an “ablated” version of the proposed approach. Moreover, the authors focus on key important evaluation metrics, such a physical consistency and diversity, which is an often overlooked and important component in generative modeling for surrogate modeling, solving inverse problems, and unconditional generation. The main downside, however, is that the authors did not further investigate the ability of their approach for uncertainty quantification, which, as they note, is another important consideration to be made for modeling physical systems. With that said, I believe they have outlined an approach that will be valuable to share with the community.", "other_comments_or_suggestions": "The position of algorithm 1 in the middle of the text is a bit strange, maybe it would be better to move it to the top of the page.", "questions_for_authors": "1.The arbitrary choice of h being Bernoulli-distributed with p=0.5 is not justified. Wouldn’t a network trained with such an assumption always “assume” the input to have roughly half of the components masked? Have you experimented with varying p through training?\n\n2. “We used FNO to construct our denoising network” What does this mean? You share the same architecture as FNO?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes ACM-FD, a novel diffusion-based surrogate model for multi-physics emulation that supports diverse tasks—including forward prediction, inverse inference, joint simulation, and function completion—within a single framework. The method extends DDPMs to functional spaces using Gaussian Process noise and introduces a masked, zero-regularized loss for flexible conditioning. It also leverages Kronecker-structured sampling for computational efficiency. Empirical results show strong performance across 24 tasks and four physics systems, outperforming task-specific neural operators in most cases. The reviewers raised a few points that are worthwhile to strengthen the work. I would like to see them included in the camera ready version. Overall, this is a well-executed and promising contribution, though some key claims would benefit from deeper empirical support.

I agree with the reviewers to accept this paper.
Decision: Accept (poster)