FUSE: Fast Unified Simulation and Estimation for PDEs
Authors: Levi E. Lingsch
Dana Grund
Siddhartha Mishra
Georgios Kissas
Venue: NeurIPS 2024 poster (2024)
Keywords: Forward and Inverse Problems
PDEs
Neural Operators
Neural Posterior Estimation
Primary area: machine_learning_for_physical_sciences
TL;DR: This work presents a framework to unify forward and inverse problems in scientific computing by optimizing a joint objective derived from operator learning.

Abstract: The joint prediction of continuous fields and statistical estimation of the underlying discrete parameters is a common problem for many physical systems, governed by PDEs. Hitherto, it has been separately addressed by employing operator learning surrogates for field prediction while using simulation-based inference (and its variants) for statistical parameter determination. Here, we argue that solving both problems within the same framework can lead to consistent gains in accuracy and robustness. To this end, we propose a novel and flexible formulation of the operator learning problem that jointly predicts continuous quantities and infers distributions of discrete parameters, thereby amortizing the cost of both the inverse and the surrogate models to a joint pre-training step. We present the capabilities of the proposed methodology for predicting continuous and discrete biomarkers in full-body haemodynamics simulations under different levels of missing information. We also consider a test case for atmospheric large-eddy simulation of a two-dimensional dry cold bubble, where we infer both continuous time-series and information about the system's conditions. We present comparisons against different baselines to showcase significantly increased accuracy in both the inverse and the surrogate tasks.

FUSE: Fast Unified Simulation and Estimation for PDEs
Levi E. Lingsch Seminar for Applied Mathematics & AI Center, ETH Zurich levi. lingschhai. ethz. ch
Dana Grund
Institute for Atmospheric and Climate Science, ETH Zurich dana. dana. graaaderundetz..... ch
Siddhartha Mishra
Georgios Kíssas
Seminar for Applied Mathematics & Al Center, ETH Zurich siddhartha mwwsmaaeeeeaaee ch
AI Center ETH Zurich Stissas@ai.ethz. ch
Abstract
The joint prediction of continuous fields and statistical estimation of the underlying discrete parameters is a common problem for many physical systems governed by PDEs. Hitherto, it has been separately addressed by employing operator learning surrogates for field prediction while using simulation-bail based inference (and its variants) for statistical parameter determination. Here, argue that solving both problems within the same framework can lead to consistent gains in accuracy and robustness. To this end, propose novel and flexible formulation of the operator learning problem that allows jointly predicting inferring distributions of finit-dimeensional parameters and thus amortizing the cost of both the inverse and the surrogate models to a joint training step. We present the capabilities of the proposed methodology for predicting continuous and discrete biomarkers in full-body haemodynamics simulations under different levels of missing information, and test case for atmospheric large-eddy simulation wwo-dimensional dry cold bubble, predicting continuous time-series measurements from inferred system conditions. For both cases, we present comparisons against different baselines to demonstrate significantly increased accuracy in both the inverse and surrogate tasks.
dimnntdedddesiinall quantities and
Introduction
Partial Differential Equations (PDEs) describe the propagation of system conditions for very wide range of physical systems. Parametric PDEs consider different system conditions as well as an underlying solution operator characterized by set of miite-dimensional parameters. Traditional numerical methods based on different discretization schemes such as Finite Differences, Finite Volumes, and Finite Elements have been developed along with fast and parallelizable implementations to tackle complex problems, such as atmospheric modeling and cardiovascular biomechanics. For parametric PDEs, these methods define maps from the underlying set of parameters, which describe the dynamics and the boundary/initial conditions and which we assume to be finiteddimensional here, physical quantities such velocity or pressure that are continuous in the spatio-temporal domain. Despite their successful application, there still exist well-known drawback: of traditional solvers. To describe a particular physical phenomenon, PDE parameters and solvers need to be calibrated on precise conditions that are not known priori and cannot easily be measured in realistic applications. Therefore, iterative and thus expensive calibration procedures are considered in the cases where the parameters and conditions are inferred from data [2]. Even after the solvers are calibrated, an
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

ensemble of solutions needs to be generated to account for uncertainties in the model parameters or assess the sensitivity of the solution to different parameters, which are computationally prohibitive downstream tasks [43].
Related work A variety of deep learning algorithms have recently been proposed for scientific application to PDEs, broadly categorized into surrogate and inverse modeling algorithms, to either reduce the computational time of complex forward simulations or infer missing finiteddimensional information from data to calibrate simulator to precise conditions.
Surrogate learning is paradigm for accelerating computations. Hence, the cost of evaluating continuous quantities is amortized to an offline training stage. For functional data, the so-called Neural Operators [28, 3] generalizes to different conditions and discretizations of complex system (resolution invariance). A multitude of operator learning approaches has been designed [37, 27, 50, 21, 44, 33, 32, 17], such as the Fourier Neural Operator (FNO) [31]. However, priori, neural operators are not designed to process maps between finite-dimensional and continuous quantities. different surrogate modeling strategy constructed by learning map between PDE parameters and solutions, as in traditional reduced-order model (ROM) approaches [16, 8, 191, and their deep learning counterparts [23, 10], including Generative Adversarial ROM (GAROM) [10]. In addition, Gaussian processes and kriging have been revisited in the context of PDE emulation [15, 61J.
Amortizing the cost of parameter inference has also been widely explored in the literature of generative modeling and Simulation- -aaseatto--BBsed Inference (SBI). In SBI, Invertible Neural Networks are combined with discrete Normalizing Flows develop Neural Posterior Estimation [11, 12, 24, 45] or continuous Normalizing Flows in Flow Matching Posterior Estimation (FMPE) [60, 35). However, they commonly rely on expensive physical solver sample continuous predictions, and hindered by the use of approximate physical models, Gaussian distributions, or a limited sample size.
Approaches that attempt to jointly infer parameters and learn a surrogate are much rarer in the literature. Many are built on Variational Autoencoders (VAEs) [57], in finite [25] or infinite [51] dimensions. They inherently assume underlying bijective relation between the functional and scalar quantities or are restricted to Gaussian latent representations, which limits their applicability to nonlinear problems. Among them, In VArt [57] extends a deterministic mncoder-decode, pair with variational latent encoder. However, In VAEr is primarily constructed for investigating identifiability of parameters in systems of PDEs, and may not be suitable for learning uncertainty propagation from parameters to continuous fields. The conditional Variational Neural Operator (cVANO) [26] makes of the resolution invariance of neural operators, based on [51]. eVANO relies VAE to learn low-dimensional manifold of the system constrained by fiite-dimensional parameters, but is restricted Gaussian approximations which are not suitable for problems which lack bijectivity. As alternative approaches, Simformer [14] estimates both finite-dimensional and function- valued parameters using transformers within the SBI framework, and PAGP [62] leverages the expressive power of Gaussian Processes. However, both are not suited for functional input due to quadratic growth with the input size. Finally, OpFlow [53] brings together operator learning and normalizing flows for quantifying uncertainty in output functions, but it lacks interpretability and inference on the latent space.
Our contribution Based on the experience and limitations of the aforementioned approaches, we formulate the following requirements for a unified forward-inverse framework relating functional quantities to known space of finitedimmensionl parameters: First, the interpretability provided by the given parameterization should be retained in both the forward and inverse tasks as well as their combination. Then, both the forward and inverse model should be discretization invariant with respect all functional variables. Finally, a general probabilistic representation should be chosen for the latent parameter space, allowing for arbitrary parameter distributions. Following these guidelines, the main contributions of our paper are the following.
We propose Fast Unified Simulation and Estimation for PDEs (FUSE), a rigorous framework for unified inverse and forward problems for parametric PDEs. The FUSE framework, illustrated in Figure allows combine different forward and inverse models. and we choose to instantiate it with Fourier Neural Operators (FNOs) and Flow-Matching Posterior Estimation (FMPE) for the experiments in this work.
We formulate mathematical framework with a unified objective for the forward and inverse problems, which allows us to assess how uncertainty in the inverse problem propagates

---

through the forward problem (propagated uncertainry) and evaluate both models together to avoid nonlinear enror amplification at model concatenation.
forward We showcase how to adapt forward surrogate and inverse estimation models to fit the FUSE framework, mapping between finite and infinite-dimensee dimensional spaces. In particular, we extend FNO with custom lifting take fiite-diimeniional inputs, and equip FMPE with an FNO-based encoding to allow for functional conditional information.
Our experiments show that our implementation of FUSE overcomes struggles baselines (cVANO, GAROM, In VArt) and an ablation (U-Net) on two complex and realistic PDE examples, pulse wave propagation (PWP) in the human arterial network and an atmospheric cold bubble (ACB). The experiments exemplify its ability for accurate and fast surrogate modeling, parameter inference, out-of-distribution generalization, and the flexibility to handle different levels of input information.
To our knowledge, the extensions we present to FNO and FMPE have not been explored in the literature yet. We would like emphasize that chose base ourselves on these methods to illustrate the FUSE framework, but other forward and inverse methods may be more suitable for particular test cases.
co-dim.
fin.-dim. & E R
co-dim. $
Inverse Problem
Forward Problem
Band Limited
Inverse Fouriar
Lifting
Operator + Projection
Operator
Transform
Lifting
Transform
M mm
E
Training: (u, E*)
Evaluation: (u, ?)
Figure 1: FUSE models a posterior distribution over finite-dimensional parameters & given infinite- dimensional functions with d, components (channels). It learns other continuous functions s with de channels from the parameters Ç. Land-limited Fourier transforms and lifting operator act as bridge between finite and infinite dimensions for the forward problem. Likewise, as inference models such as FMPE or NPE require fixed-size inputs, the inverse operator layers are conjoined with band-limited Fourier transform to learn fixed-size representation of the input function.
2 Methods
Notation and Assumptions Let U C C(X, Rd.) and S C C(Y,RR)) be spaces of continuous functions on compact domains X C Rd and Y C Rdd, respectively. and let =\t S R" be space of finiteddimensional parameters. For a map F A - B, where we take and B to be among the spaces 41 S, or E, and probability measure E Prob(AA), we denote by F#F E Prob(B) the push-forward measure that expresses uncertainty on set B C by the propagation of through F, defined as F ( π (F-+(B)) Further, we assume all metrics be the total variation metric, and denote them by both on Prob(S) and Prob(E). In general, we assume all maps be Lipschitz continuous, and we omit Lipschitz constants in inequalities (details are provided the Appendix).
measures in this paper to
Problem Formulation Consider the setting of a parametric PDE with forward solution mapping g: & 8, where E is a vector fnite-dimensional parameters and & E S is function-valued output. Since it is the goal to calibrate the input parameters, omit any functional inputs to G. such
3

---

as initial or boundary conditions, and assume they are kept constant or encoded in the parameters ç. Given the forward operator G. the forward model uncertainty is quantified by propagating a given distribution of parameters p E Prob(=) through the model, i.e. evaluating the push-forward measure G#p. In practice, the parameters Ç are not available and need to be inferred from indirect measurements u E U, which we assume to be functions in space or time (e.g., time series), and, in general, u f The extended solution operator 9 u + 8 then maps between functional measurements and functional model output, omitting the intermediate parameter space. Since the measurements are impacted by the uncertain parameters Ç. the function inputs are as well equipped with uncertainty and represented by the measure μ E Prob(Uf). Finally, the inverse problem consists in estimating the distribution p(E u)) of parameters given measurements The corresponding uncertainty on the predicted outcomes s is then given by the propagated uncertainty #
Unified Objective Given an approximate forward operator 9 = & between function spaces, and an estimated distribution μr = yu of input functions u, parameterized by and Φ respectively, we use triangle inequality to observe that
d( #1o mmm) S dl #1o ( + d( #v 9wm)
(1)
Measure matching
Operator learning
Thus, we found a unified objective consisting of two steps corresponding to the two terms in the right hand side (rhs) of Equation (1). In the measure matching step, the objective amounts to learn an approximation م of the measure Je, whereas in the operator learning step, the objective is to learn a Neural Operator Go that approximates the underlying ground truth operator G.
Our reformulation of operator learning naturally fits into the aim of this paper to propose a method that can act an operator surrogate (forward problem, operator learning objective) well as performing parameter inference by minimizing distances on measures (inverse problem, measure matching objective), unifying the apparently unrelated problems of surrogate modeling and inference.
Forward Problem: Operator Learning The operator 9 resembles the solution operator to PDE that maps function inputs such as initial and boundary conditions, coefficients, sources etc to (observables of) the solution of the PDE. It the goal of supervised operator learning [29] to learn this type of operator by minimizing the operator learning objective in (I). This objective bound by supervised learning on the parameter space E,
parameterized Neural Operator (NO) 9 on (training) distribution μ E Prob(l4)
d( #1 G#u) S d(G G4p);
(2)
and further by the term (Appendix A.6),
(0) = / (10 ((E))l-dd())
The loss is approximated by training samples of the form ( 9(5*)) sampled from an underlying data distribution م E Prob(E). In order to use a NO on finiteddimensional inputs, we define the band-limited lifting h (f), composed of lifting increasing the dimension of the parameters and an inverse Fourier transform. We then choose to instantiate G based on Fourier Neural Operator (FNO) [31] layers x implemented with discrete spectral evaluations in [34] to handle irregularly sampled measurements for the output function,
(C(( = Qdh o K (
where Qth is learnable map that projects the channels to the dimensions of the output function and 0 = [0, 82 fg] are the trainable parameters.
Inverse Problem: Measure Matching We would like to approximate the true posterior measure p(Elu) given measurements u by o (Elu). This task is equivalent to minimizing the measure matching objective in (1) since (Appendix A.7)
d(@r #u (o S (Elu,
(5)
To minimize the latter distance, we adapt Flow-Matching Posterior Estimation (FMPE) to handle function-valued valued conditional inputs u. FMPE trains flow function that maps samples from standard

---

normal distribution to those from the target distribution p(Elu). Unlike diffusion models [54, 20, 58], which require multiple steps, FMPE achieves this transformation in single step, enabling faster training and evaluation. Both approaches are similar, but differ in their probability paths: diffusion models use a diffusion-based path, while FMPE employs an optimal transport path, leading to greater regularity and improved performance in low data settings.
The function input u is handed to the FMPE flow parameterized by 1 (5) as finite-dimensions conditional information ú. We encode the infinite-dimensionall into û T as an adapted FNO, mimicking the forward surrogate. We project the functions to the parameters using a band- limited forward Fourier transform à to create fixed size latent representation of a function regardless of the input dimensions, subsequently projecting the large latent dimension to the smaller dimension v of & by FMPE,
u T (u) = h o K o po (u):
where pe is a lifting layer and 1 a concatenation of FNO layers. For the detailed formulation of the FMPE loss, we refer to the original reference [60] and equation (32) in Appendix A.8, and formulate it simply as
(の = (FMP
(7)
where o 100 Φ1 02] are the joint trainable parameters of the flow itself and the FNO-based conditional information. The flow is then trained on data of the form {(u, (.))==
Unified Training and Evaluation The proposed methodology FUSE obtains the optimally trained parameters 0 and & by minimizing the respective objectives of Equations (3) and (7),
Cruse (0, Φ) /forward (6) + FUSE (+) (0) + C2(Ø),
claverse
thus minimizing the joint objective (1). For all experiments presented here, the two FUSE objectives are decoupled during training. implying that for each training batch, L1 and L2 are calculated and backpropagation is performed separately over each set of network parameters. This strategy aids the training, allowing us avoid hyperparameter tuning of the loss function. It is possible to backpropagate losses through the entire network as discussed in Appendix 1. I; however, this does not result in improvements evaluation time, further supporting may be unified at evaluation time to draw connections between forward and inverse problems.
claim that decoupled objectives
When employing the trained FUSE model, we assume that measurements u are available and it is the goal to infer both the distribution parameters & that best fit the data and the distribution of function outputs that would result from those parameters. We obtain M parameter samples from the trained FMPE model as S, (E)()) i 1,.... M, and pass these to the FNO forward surrogate to obtain M samples of the simulated outputs 8; = (E)), i 1, ..., M, where we assume M sufficiently large to adequately sample the parameter distribution. We hence obtain an ensemble of estimates with mean prediction both model parts can also be evaluated separately, it is this propagated uncertainty on that offers most insight the effect of parametric uncertainty within the emulated models.
TZ Σ/ 8; and standard deviation o, ( ΣM (s; 3)2)2/2, While
Experiments
For evaluation, we are given testing data of the form (u, (*) obtained by numerical simulation, where we will refer to g" as the data-generating purameters. Based on measurements u, the inverse FMPE model provides samples of the distribution of discrete parameters &., that are compared to the true data- fata-generatiin parameters ?* using the Continuous Ranked Probability Score (CRPS). The performance of the deterministic forward prediction is measured with the relative L and L errors over the continuous data and predictions. We compare FUSE against cVANO [26), VA [57], and GAROM [10], and we consider an ablation where we substitute the FNO layers with UNet layers [47] in the FUSE model and leave out the Fourier transforms. In addition, we provide an ablation using denoising diffusion probabilistic model (DDPM) [20] instead of FMPE in Table 5, and examine FMPE's performance with varying data sizes in Table 6. For details on the training and evaluation metrics, please refer A.2. PDE Test Cases As test cases, we consider simulation of the pulse wave propagation (PWP) in the human arterial network and one of an atmospheric cold bubble (ACB). Both experiments are
5

---

described by a set of PDEs whose solutions are fully parameterized by Unite-dimensione parameters encoding model properties, as well as boundary and initial conditions. In both of these experiments. to replicate a patient or downburst observed in the real world, the solver needs to be calibrated for precise conditions, e.g. a specific patient or atmospherie conditions, and forward uncertainty quantification of the calibrated predictions are of interest. Details on both cases are provided in Appendices A.3 and A.4.
Atmospheric Cold Bubble (ACB) In atmospheric modeling, the two-dimensional dry cold bub- ble [56] is well-known test case for numerical simulations resembling downbursts, which cause extreme surface winds in thunderstorms. Within a domain of neutral atmospheric stability without background wind, an elliptic cold air anomaly is prescribed and initiates a turbulent flow as it sinks to the ground, see Figure 12 for an example. Time series of horizontal and vertical velocity, and w, are recorded at eight sensors placed inside the domain. Similar data is obtained by weather stations (z R 2 m) or turbulence flux towers (z R 10 50 m), experienced by wind turbines or high-rise buildings (z R 100 m), or with unmanned aerial vehicles. The PDE model PyCLES [42] used for simulation is parameterized by the turbulent eddy viscosity V and diffusivity D; as model parameters, as well as four parameters describing the amplitude and shape of the initial cold perturbation (a, Tr; 2r, ze). The target task for FUSE here is to calibrate these parameters & E R to time series measurements u [0, T] + R220 at ten locations for horizontal and vertical velocity each, where the input measurements used for calibration and the model output used for prediction and uncertainty quantification share the same space Ut = S of time series.
Pulse Wave Propagation (PWP) in the Human Body The pulse wave propagation in the cardio- vascular system contains a great deal of information regarding the health of an individual. For this reason, there are efforts to measure and leverage PWP in both wearable devices, e.g. smart-ware watches, and clinical medicine. Different systemic parameters & E IR such as stroke volume (SV), heart rate (HR), and patient age have been shown to affect the morphology of pulse waves [7]. These are used to parameterize pulse wave time series of pressure, velocity, and photoplethhsmoograhhi (PPG) at 13 locations of systemic arteries of the human cardiovascular system through reduced order PDE model in the dataset published by [7]. See Tables 7 and in Section A.3 for the full set of parameters and locations. In clinical applications, only measurements at casily accessible locations (such as the wrist) are available, while it is the goal to predict the pulse signal s at other locations of interest from the inferred parameters g. The space of input functions 2/ hence differs from the space of output functions S in this test case. In order to account for different clinical scenarios, we train the FUSE model using random masking of locations and evaluate it on different levels of available input information,
Level 1. Perfect information:
Pressure, velocity, and PPG at all locations, Level 2. Intensive care unit information: Pressure, velocity, and PPG at the wrist,
Level 3. Minimal information: PPG at the fingertip.
4 Results
Collected results for all experiments are summarized in Table 1. While the main tasks of the FUSE model are evaluated for both test cases, we focus on ACB to exemplify its generalization properties and on PWP for different levels of input information. We would like to emphasize that the goal of uncertainty representation here is to capture parametric uncertainty inherent to the data-generating numerical model and its parameterizations. Our model does not provide a notion of uncertainty the predictions due to imperfect training of the neural networks. Given that these errors are generally low (Table 1), we are confident to interpret all spread on E and s given by FUSE as parametric uncertainty in the data-generating model. In particular, we expect the true posterior distributions to have positive spread, hence a prediction with small ensemble spread is not necessarily accurate.
Inverse Problem Given time series measurements u. the inverse model samples from the distribution of parameters, & ~ (f(u). with more samples in areas of the parameter space that are more likely to match the data-generating parameters E*. We find that FUSE outperforms the other methods when sufficient input information is given. We observe that VAErt experiences posterior collapse for PWP in this setting, which is represented in high CRPS (Figure A.9). Histograms reveal that FUSE captures the expected dependencies of the data on the parameters (Figures A. A.2), such as wider distributions for scarcer information in PWP. In the ACB case, sharp Dikac-like estimates are obtained

---

for most of the samples (Figure A. 14). However, a good fit for the numerical model parameters seems rely on an accurate estimate of the initial condition (Figure A.16).
Forward Problem The surrogate (g) is tasked to assess the ability of FUSE to predict the continuous from the data-generating parameters 5" FUSE consistantly outperforms all other methods in both test cases. In particular, FUSE is able to reproduce very nonlinear dynamics and sharp velocity gradients in the ACB case (Figure A.15). Potential errors mostly lie the bulk velocity of the anomaly, resulting in a stretching or squeezing of the time series at later times. As expected, predictions are slightly worse further away from the perturbation location, where the flow has become more turbulent, especially for the vertical velocity which exhibits strong oscillations (Figure A.) For the PWP, continuous predictions based on the true parameters have an LI error of well below 0.3% at all locations (Figures A.6, A.7). Smaller arteries generally exhibit slightly higher error variance, which fits the physical problem interpretation, as smaller diameters show more variations to pressure than larger system arteries.
Unified Uncertainty Propagation Given an input u, the concatenated evaluation of the inverse and forward parts of FUSE provides a distribution over the output functions 8, by first sampling parameters & and then evaluating the forward emulator on this ensemble, yielding continuous samples 8,. This can be interpreted as the uncertainty introduced by simulating the data with the parameterized model FUSE was trained on: Uncertainty is introduced as the parameters are not fully identifiable in the inverse step, and is propagated and expanded onto the continuous output through the ensemble predictions on the parameter samples. Again, FUSE outperforms all other models significantly in all experiments in the 1 and 1,2 error over the mean predictions s. Figures 2 and 3 show how FUSE captures both smooth and rough structures, even on the sample performs worst on. It only exhibits slight difficulties in predicting a constant velocity close to zero for ACB. At each location of the ACB, the L error obtained with the propagated ensemble mean 3 is comparable to the error by evaluating only the forward model, s (E*) (Figure A.)1b). An inspection of the predictions for all locations for the PWP case shows that the true time series lie within one standard deviation of the FUSE ensemble mean for the median sample, and within the min-max uncertainty bounds for the worst sample (Figures A.3, A.4). We observe that the accurate prediction of model parameters correlates to accurate predictions of pressures (Figure A.11).
Taa
Towrdd
Toari
(a) FUSE
(b) UNet
(c) InVAErt
(d) cVANO
(c) GAROM
Figure 2: ACB, propagated uncertainty: Comparison to the benchmark models on the horizontal velocities at location 1, showing the unified ensemble predictions for the sample where FUSE has the largest L error, i.e. a worst-case scenario. *GAROM predicts the velocity from the true parameters, while all other models predict pressure from the posterior distribution over parameters given continuous inputs.
Morker
Tour
FUSE
(b) UNet
(c) InVeE
(d) <VANO
(c) GAROM
Figure 3: PWP. propagated uncertainty: Comparison to the benchmark models predicting the Sup. Middle Cerebral pressure and uncertainty for the sample where FUSE has the largest L error, i.e. a worst-case scenario. *see Figure 2.
7

---

Sensitivity Analysis Based on the good approximative properties showcased above, the forward operator 9 of FUSE can be used as a fast surrogate model assess the sensitivity of the underlying PDE to varying the parameters Ç. Such analysis is usually constrained to small sample size due to the large computational costs, and the dense sampling enabled by the surrogate helps to explore the parameter space for parameters that optimally fit patient data in PWP, or exhibit extreme winds in ACB. Varying one parameter at time, while keeping all others fixed at their default value, exhibits the single effect on the data (fingerprint). For ACB, an increased amplitude of the cold anomaly mainly results in a speed-up and hence squeezing of the time series, while other parameters have more nonlinear effects (Figure A.17). As the FUSE model provides access to the full velocity fields, only numerical simulations reveal that these sharp sensitivities are mainly caused by certain features (eddies) of the flow reaching missing sensor depending validation of this sensitivity analysis for an estimate of the maximal horizontal velocity over the time series is provided in Figure 4 for a pairwise fingerprint with 10, 000 FUSE samples, with consistently low errors compared to 100 numerical samples. Only for one sample with a weak and small perturbation, the signal is not captured accurately. An additional evaluation of 60 samples the margins of the parameter space shows that the peak velocities are consistently represented well (Figures A. 19).). For PWP, the sensitivity results (Figure A.1 are consistent with observations reported in the literature [7].
the parameter value. A
186
3.21
50
150
0.41
angituste
196
1540
10.0
10.
3,07
50
58
Jaflaeedy
Figure 4: ACB, sensitivity analysis and generalization: Validation of the FUSE model against numeric simulations on peak horizontal velocities u at location (x km, m). Samples above the dashed line correspond to amplitudes larger than seen during training. The parameter resolution is 100 x 100 for FUSE, and (10 + 10) x 10 for the numerical samples. Figure continued in the appendix, Figure A.18.
Out of Distribution Generalization for ACB We test the generalization capabilities the FUSE forward model test samples with amplitudes larger than in training in the pairwise sensitivity setup (Figure 4). The FUSE model is able to capture the shape of the pairwise parameter dependencies but struggles to locate the sharp dependence of the local peak velocity on the eddy diffusivity. Averaged errors in the out of distribution range are shown in Table 2, with FUSE performing clearly superior to all other models.
Levels of Available Information for PWP The case of missing input information is modeled by masking certain input components when evaluating the models. While FUSE shows regular performance comparable with the baselines on levels 2 and 3 of missing information, cVANO be better suited for this setting (Table 1), probably due the assumption that the latent dimensions are uncorrelated. As a sanity check, FUSE samples the prior distribution the training data when evaluated without any conditional input information u (Figure A.8). When the available input measurement locations of the PWP experiment are known a priori, it is possible to get more accurate results when training FUSE without masking. For example, training to predict pressures and parameters from only the PPG data at the fingertip only, without masking, results in CRPS of 4.28 x 10-2 and relative L' error of 3.6%, roughly half the error of the masked model. the measurement data is fixed, predictions without masking are much more suitable for analysis.
5 Discussion
Summary We propose FUSE, a framework unifying surrogate modeling and parameter identification for parametric PDEs by bridging operator learning with flow-matching posterior estimation. Both are represented the FUSE objective through a deterministic forward loss and probabilistic inverse loss, respectively. The joint architecture allows for inverse estimation of scalar parameters given

---

Table Performance of FUSE and the baseline models in estimating parameters from continuous inputs quantified by CRPS, and predicting time series data s, quantified by relative L1 and L2 error. Here, "True parameters" evaluates the forward model part only, and "Estimated parameters" and levels one to three evaluates the sample mean & predicted by the unified model.
Model
Experimental Setup
CRPSx 10
Rel. L1 Error
Rel. Ly Error
FUSE (ours)
True Parameters
0.13 1 0.05 %
0.19 1 0.08
Level I
1.31 0.69
0.86 1 0.93 %
0.90 - 0,92 %
Pulse Wave Propagation
Level 2
3.18
1.40
3,83 = 2.66 %
3.87 + 2.64
Level 3
8.48 + 2.55
6.56 3.36
7.05 3.39
UNet (ablation)
True Parameters
1.38 = 0.34 %
2,00 + 0.48
Level 1
3.79 + 0.97
3.81 2.03%
4.25 1.92
Level 2
5.75 + 1.32
6,79 - 3.77 %
7.19 + 3.68
Level 3
12.72 + 2.65
7.83 + 2.64 %
8.78 + 2.87
InVAErt
True Parameters
3,44 +: 0.52 %
4.21 1.60
Level 1
11.62 2.39
8,28 +: 4.78 %
8,94 1 4.91 %
Level 2
16.88 + 3.67
7.85 + 2.80 %
8.64 + 2.99
Level 3
4.35
8.37 2.73 %
9.48 2.98
eVANO
True Parameters
7.49 1 2.60 %
8.51 1. 2.58
Level I
5.61 0.24
7.54 2.52
8.52 2.53
Level 2
4.00 + 2.22
8.42 - 3.11 %
9.72 + 3.23
Level 3
3.66 + 0.76
10.79 = 3.60 %
12.54 + 4.05 %
GAROM
True Parameters
0.70 = 0.13 %
1.04 + 0.22
Levels 1-3
FUSE (ours)
True Parameters Estimated Parameters
0.41
0.45 %
0.80 0.89
Atm. Cold Bubble
1.84
0.58 0.59 %
1.13
1.11
UNet (ablation)
True Parameters Estimated Parameters
1.70 0.90 %
2.86
1.55
1: 2.67
2.13 1.16
3.69
1.94
InVrt
True Parameters
1.11 0.85 %
1.94 1.31
Estimated Parameters
6.65 + 9.05
1.17 = 1.35 %
2.88 + 2.04
eVANO
True Parameters
5.32 1.57 %
8.26 = 2.44%
Estimated Parameters
19.84 4.73
8.77 = 3.04 %
12.33 + 4.00%
GAROM
True Parameters
3.99 1.67 %
6.35 2.43
Estimated Parameters
Table 2: Performance of FUSE on out-of-distribution samples for ACB, as described Figure 4, following the format of Table 1.
Model
Experimental Setup
10²
Rel. Lı Error
Rel. La Error
Out-of-distribution (ACB)
Fuse (ours)
True Parameters
1,83 = 1.10 %
3.15 : 1.80 %
Estimated Parameters
3.13 + 1.13
1.63 = 0.78 %
2.77 + 1.28 %
UNet (ablation)
True Parameters
2.78 + 0.32 %
4.37 1: 0.43 %
Estimated Parameters
12.36 5.47
3.43 1 0.40
5.40 1: 0.58 %
InVAErt
True Parameters
4,48 1 2.65 %
6.49 1: 3.26 %
Estimated Parameters
18.80 1 19.09
4.31 1: 1.13
6.92 1. 1.83 %
eVANO
True Parameters
9.31 = 1.22 %
13.74 + 1.54 % 1.49%
Estimated Parameters
24.76 1.45
15.35 1.28 %
21.20
GAROM
True Parameters
7.22 = 1.00
10.96 + 1.22 %
Estimated Parameters

---

continuous measurements u, as well as forward predictions of (other) continuous measurements given Ç. Inheriting from the properties of its components, FUSE discretization invariant with respect to the continuous inputs and outputs, and can represent arbitrary distributions in the fully interpretable parameter space. As this latent space is defined through a known parameterization of the underlying PDE, FUSE: allows for accurate calibration and evaluation of a numerical solver, as well as performing downstream tasks such as parameter uncertainty quantification. On the parametric PDEs of pulse wave propagation (PWP) in the human cardiovascular system and an atmospheric cold bubble (ACB) with time series measurements, FUSE consistently outperforms four baseline methods designed to perform similar tasks. It also shows good out- -fdistiiiititttoi generalization and great flexibility when trained and evaluated on different levels of available input information.
Connection to Existing Methods The proposed methodology and the adapted implementation of the FNO and FMPE offer the following advantages over the existing methods for joint parameter estimation and forward emulation mentioned in the introduction. By incorporating Neural Operators in both the forward and inverse parts, we ensure discretization invariance throughout the model [3]. In contrast to Variational Autoencoders, we present fully probabilistic formulation and require no restriction on the latent data distribution. This allows us to model arbitrary posterior distributions, whereas e cOANO is restricted to Gaussian distributions. Likewise, FUSE can leverage powerful, existing neural operators which improves predictions on the output functions. The experiments have shown that FUSE excels over all baselines in both test cases. In particular, e vANO and GAROM fail to capture even rough characteristics of the flow in the forward emulation for the ACB case with nonlinear dynamics, and In VAEr shows consistent bias towards the mean in the PWP task. Only in the case of missing information in the smoother PWP case, eVANO outperforms FUSE on the inverse problem. In terms of the choice of FNO in both the forward and inverse part of FUSE, the ablation with UNet layers exhibits larger uncertainties and worse performance of the predicted mean. An additional ablation, presented in the Appendix 5, shows that optimal transport probability paths perform better than the diffusion- based paths in a conditional DDPM model. Finally, Gaussian processes are often chosen for their ability handle unstructured data. We incorporate this property by using the grid-independent FNO implementation by [34].
Applicability FUSE is explicitly formulated for diniee-immniiooall parameters and function measure- ments and outputs. This setting naturally arises when calibrating a numerical solver, with training data generated synthetically by the solver itself, and was demonstrated in two test cases. However, it is very common for real measured datasets, e.g. in bioengineering involving PWP, to contain both time series data and vectors of parameters available for different patients. FUSE naturally extends to these datasets, where an underlying PDE is assumed but not formulated or simulated explicitly, including parameter inference using real data [59], precision medicine or solver calibration [46], and fingerprinting to discover parameter-disease correlations [55]. In the case that the parameters are non non-physica,, and hence not measurable, such as the numerical model parameters in the ACB case, this does of course not apply, and the main aim of solving the forward and inverse problem is to explore the solver itself. Moreover, FUSE may help to refine existing parameterizations either through disentanglement [18] or by identifying parameters with little influence on the simulated data.
[22], and more specifically when
Limitations and Future Work The FUSE framework aims to emulate a given parameterization of a PDE. If such parameterization (or corresponding dataset) is not available, VAE-based preprocessing [4] or manifold discovery [5] may be used. When applying FUSE to real measurements, the parameterization turns to be only an approximation of the dynamical system, whereas it is considered a perfect model in the experiments presented here. This structural model uncertainty has to be added to the interpretation of the unified uncertainties given by the FUSE model, as has to be for any other inverse algorithm. In practice, measurements are always associated with a measurement error, which is considerable in particular for extreme measurements. The current implementation of FUSE does not allow for errors in the continuous inputs to be taken into account, and it is left for future extension of the framework to include these into fully Bayesian formulation of the inverse model. In spatially distributed applications, such as atmospheric modeling, the parameters of interest might be themselves ppac//ime-dependent functions, such as surface properties or initial conditions [48]. Based on the scaling properties of the FMPE model for high-dimensional data such as images [35], FUSE is expected to scale well in size for high-dimensional parameter spaces, but is yet to be extended to a resolution invariian function representation in latent space. These limitations will be addressed in future research.
10

---

Acknowledgments
G.K. would like to acknowledge the support from Asuera Stiftung via the ETH Zurich Foundation.
References
[1] Alastruey, K. H. Parker, S. J. Sherwin, al. Arterial pulse wave haemodynamics. In 11th international conference on pressure surges, volume 30, pages 401-443. Virtual PiE Led t/a BHR Group Lisbon, Portugal, 2012.
[2] A. Arzani, J.-X. Wang. M. S. Sacks, and S. C. Shadden. Machine learning for cardiovascular biomechanics modeling: challenges and beyond. Annals of Biomedical Engineering, 50(6): 615-627, 2022. 1001110000111119900222222222
[3] F. Bartolucci, E. de Bezenac, B. Raonic, R. Molinaro, S. Mishra, and R. Alaifari, Representation cquivalent neural operators: a framework for alias-free operator learning. Advances in Neural Information Processing Systems, 36, 2024. URL html.
nips cc/ hatpppppe_reeeee files// www.-raatttooofeeeeeeeeeeee
[4] P. Bojanowski, Joulin, D. Lopez-Pas, and A. Szlam. Optimizing the latent space of generative networks. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 600-609 PMLR, 10-15 Jul 2018. URL attps://proceeddnngg .mw.............m............... mlr. jress/v80/bojeii jojaanossiieee 18a. html.
[5] J. Brehmer and K. Cranmer. Flows for simultaneous manifold learning and density estimation. In Advances Neural Information Processing Systems (NeurIPS), 2020. URL https://proceedinss.eeee iectiinngsnnuuiippe 0rg.6911ee77ca000000000000000000
[6] S. Čanić, J. Tambača, G. Guidoboni, A. Mikelić, C. Hartley, and D. Rosenstrauch. Modeling viscoelastic behavior of arterial walls and their interaction with pulsatile blood flow. SIAM Journal on Applied Mathematics, 67(1):164-193, 2006. 137/0001177000
[7] P. H. Charlton, J. Mariscal Harana, S. Vennin, Y. P. Chowienczyk, and J. Alastruey. Modeling arterial pulse waves in healthy aging: a database for in silico evaluation of hemodynamies and pulse wave indexes. American Journal of Physiology-HHeue and Circulatory Physiology, 317 (5)HH102--H1085 2019. doi: 10.110..2222 ww2.11/eaaapp.t.t..om 00....000112200... 2011...
[8] Y. Chen, J. Xiang. D. H. Cho, Y. Chang, G. A. Pershing, H. T. Maia, M. M. Chiaramonte, K. Carlberg, and Grinspun. CROM: Continuous reduced-order modeling of PDEs using implicit neural representations. In The Eleventh International Conference on Learning Representations, 2023. URL https: //iccr.i////////// cctvirtual//023/ 2tt///23//poaee/1112 120944.
[9] E. Cleary, A. Garbuno-Inigo. S. Lan, Schneider, and A. M. Stuart. Calibrate, em- ulate, sample. Journal of Computational Physics, 424:109716, 2021. ISSN 0021- 9991. 10. 10. 10.110066...p.220000... 109716. URL http:://www.cciecccceeett www.:eeeienccciec........... wttp://drreetttccccii/eeee... com/science/
[10] D. Coscia, N. Demo, and G. Rozza. Generative adversarial reduced order modelling. Sci Rep, 14(3826), 2024. doi: 103...033/44111 1181100///4191020220 5580002441---022--------
[11) I.. Dinh, D. Krueger, and Y. Bengio. Nice: Non-linear independent components estimation. In International Conference on Learning Representations, 2015. URL htttp:://axixx org/ abs/1410.
[12] I.. Dinh, Sohl-Dickstein, and S. Bengio. Density estimation using real NVP. In The Fifth International Conference on Learning Representations, 2017. URL https://openreviiee net/forum?idee (i-tfrrmmm
[13] Formaggia, A. Quarteroni, and Veneziani. Cardiovascular Mathematics: Modeling and simulation of the circulatory system, volume 1. Springer Science & Business Media, 2010.

---

[14] M. Gloeckler, M. Deistler, C. D. Weilbach, F. Wood, and J. H. Macke. All-in-one simulation- based inference. In Proceedings of the 41st International Conference on Machine Learning. volume 235, pages 15735-15766, PMLR, 21-27 Jul 2024. URL. https::/ppoceeeeeiieeeeee press/v235/ 2e1223//glocccci @loeckler24aa html.
mllr
[15] M. Gulian, A. Frankel, and L. Swiler. Gaussian process regression constrained by boundary value problems. Computer Methods in Applied Mechanics and Engineering, 388:114111 2022. doi.https:/dbiiorr///0. 10111//..cm..202222 11117.
[16] R. Halder, M. Damodaran, and B. C. Khoo. Deep learning based reduced order model for airfoil-gust and aeroelastic interaction. doi:10.25 14002514/11005027.
AIAA Journal, 58(4):1595-1666 8 2020.
[17] Z. Hao, Z. Wang, H. Su, C. Ying, Y. Dong, S. Liu, Z. Cheng, J. Song, and J. Zhu. GNOT: A general neural operator transformer for operator learning. In Proceedings of the 40th International Conference Machine Learning, volume 202, pages 12556-12555 PMLR, 23-29 Jul 2023. URL https: httpas///pcceeenngg///// mlr, wtt.ppppeeeee.eeeeeeeeee
html.
[18] I. Higgins, L. Matthey, A. Pal, C. P. Burgess, X. Glorot, M. M. Botvinick, S. Mohamed, and A. Lerchner. beta- VAE: Learning basic visual concepts with a constrained variational framework. In The Fifth International Conference on Learning Representations, volume 3, 2017. URL https: http:///oppeeeeeeeeeeeeee net/ ftt.
[19] S. Hijazi, M. Freitag, and N. Landwehr. POD-galerkin reduced order models and physics- informed neural networks for solving inverse problems for the navier-stokes equations. Adv. Model. and Simul. in Eng. Sci., 10(5):5, 03 2023. doi: 060.1111166640323222000222222
[20] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pages 6840-6851. Curran Asso- ciates, Inc., 2020. URL https: http:::s//ppppss..s.... mips. httppssccpppppe__a___ supper_files htllle//paperr222///eas (cbbcfecc58.aff00777111101000111 www.t//aaaa--btaaaat....... Abstract......... html.
[21] Jin, Meng, and L. Lu. MIONet: Learning multiple-input operators via tensor product. SIAM Journal Scientific Computing, 44()):3490-A3514, 2022. 46::10.1137/22M147777
[22] A. E. W. Johnson, T. J. Pollard, L. Shen, H. L. Li-wei, M. Feng, M. Ghassemi, B. Moody, Szolovits, L. A. Celi, and R. G. Mark. MIMIC-III, a freely accessible critical care database. Scientific data, 3:160035, 2016. doi:10. 10....0///daaa.0000.0 2016.35.
[23] B. Kim, V. C. Azevedo, Thuerey, T. Kim, M. Gross, and B. Solenthaler. Deep fluids: A generative network for parameterized fluid simulations. Computer graphics forum, 38(2):59-70, 2019. doi:10. 10.11111cgf..r
[24] D. P. Kingma and P. Dhariwal. Glow: Generative flow with invertible 1xl convolutions. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Asso- ciates, Inc., 2018. URL https: //papers../p/e..eee nips wwtppppppp.....c.cppcp htcppaperfilessseeeeeeeeepesseee files/ ftiles//aaprr2218//aast Abstraat..bbattt................ .htll.
[25] D. P. Kingma and M. Welling, Auto-encoding variational bayes. In International Conference on Learning Representations, 2013. URL pttttps://axxi...oo wwwooogg...o.oooooooooooooooeo.. wtbs//112..t.. 1112.6114.
[26] G. Kissas. Towards Digital Twins for Cardiovascular Flows: A Hybrid Machine Learning and Computational Fluid Dynamics Approach. PhD thesis, University of Pennsylvania, 2023,
[27] G. Kissas, J. H. Seidman, L. F. Guilhoto, V. M. Preciado, G. J. Pappas, and Perdikaris. Learning operators with coupled attention. J. Mach. Learn. Res., 23(1), Jan. 2022. URL https://dll acm. org/ ht///oi////0 55555/586588 055158866888338804...
[28] N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, Bhattacharya, Stuart, and A. Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89): 2023. URL http:// htlr//ml.ooo .wwall.org//ioo www.erorgggppeppee ptggpapers//22
2114. html.
12

---

[29] S. Lanthaler, S. Mishra, and G. E. Karniadakis, Error estimates for DeepONets: A deep learning framework in infinite dimensions. Transactions of Mathematics and Its Applications, 6(1): tnac001, 2022. doi: 10. 1093/mmatm//nac0001...
[30] M. Letafati, Ali, and M. Latva-aho. Conditional denoising diffusion probabilistic models for data reconstruction enhancement in wireless communications, 2024. URL https: //aaivv../ org org/abs/2310 19460.
[31] Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, Bhattacharya, A. Stuart, and Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, volume 2010.08895, 2021. URL atttps:/iiet http//iclrc.cceiiiiee cc/virtual/ 2021/post 0021/poster/300 /3281.
[32] Z. Li, K. Meidani, and A. Farimani. Transformer for partial differential equations' operator learning. Transactions on Machine Learning Research, 2023. ISSN 2835-8856, URL https:
/ //openreview net cest/forummid E
[33] Z. Li. D. Shu, and A. Barati Farimani. Scalable transformer for pde surrogate modeling. In Advances in Neural Information Processing Systems, volume 36, pages 28010-28039, Cur- ran Associates, Inc., 2023. URL https:///apperre nips cc/paper_filee/ppeeeeeeeeeeeeeee wttps//aapeeeeee/223. mash/590daffee (0000af4449eeeffffooo077ataaaa33 Abstract Conf erence html.
[34] I.. E. Lingsch, M. Y. Michelis, E. De Bezenac, S. M. Perera,
Katzschmann, and S. Mishra. Beyond regular grids: Fourier-based neural operators on arbitrary domains. In Proceedings of the 41st International Conference on Machine Learming, volume 235, pages 30610-30629, PMLR, 21-27 Jul 2024. URL https:/ httpee///coceeeeeeeeeeeo mlr wwwppppessseeee wvsss///v25////// haip///nnnnnhhaaa.m html.
[35] Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling. In International Conference on Learning Representations, 2023. URL https: / //iclr. wttp:cccacvc/v/22/.02. /2tp///0////ppt/113
[36] Liu, C. Zeman,, S. L. Sørland, and C. Schär. Systematic Calibration Model: Application Over Tropical Atlantic. Journal of Geophysical Research: Atmospheres, 127(23), 2022. doi: 1000022022200033300.
Resolving
[37] L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218-229, Mar. 2021. 1t20000088//225602110000022222
(38] A. Moore. Investigating the Near-Surface Wind Fields of Downbursts using Series of High- Resolution Idealized Simulations. Weather and Forecasting, 2024. 0164.1.
[39] L. Orf, E. Kantor, and E. Savory. Simulation of a downburst-producing thunderstorm using very high-resolution three-dimensional cloud model. Journal of Wind Engineering and Industrial Aerodynamics, 2012. 10.160.0016/....... 10.101/f..eeiia2222..220.
[40] A. Parodi, M. Lagasio, M. Maugeri, B. Turato, and W. Gallus. Observational and Modelling Study of a Major Downburst Event in Liguria: The 14 October 2016 Case. Atnosphere, 10(12): 788, 2019. doi:10.3390/amm 1012000000 10120788.
[41] O. Pauluis. Thermodynamic Consistency of the Anelastic Approximation for Moist Atmo- sphere. Journal of the Atmospheric Sciences, 65(8), 2008. do0.:00.175/20077/2227..1
[42] K. G. Pressel, C. M. Kaul, T. Schneider, Z. Tan, and S. Mishra. Large-eddy simulation in an anelastic framework with closed water and entropy balances. Journal of Advances in Modeling Earth Systems, 7(3):1425-1456 2015. ISSN 1942-2466. doi: 10. 00000/00 URL wwley........................... com/ htld/doi//a http///i//aa////////////ee /01 1000 10000//00150000056.
[43] A. Quarteroni, A. Manzoni, and Negri. Reduced basis methods for partial differential equations: introduction, volume 92. Springer, 2015.
13

---

[44] B. Raonic, R. Molinaro, T. De Ryck, T. Rohner, F. Bartolucci, R. Alaifari, S. Mishra. and E. de Bézenac. Convolutional neural operators for robust and accurate learning of pdes. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36. Curran Asso- ciates, Inc., 2023. URL https: /ttp:///pppes..pepp/ nips wwtpppcrrrpaaaaaaeeeee files wwtp.//sspppp//223.111// ppppp/p222//aash/ S haw.eeeebbbbbasasaaatttte Abstract- Conference Eestract-Conferencee. html.
[45] D. Rezende and S. Mohamed. Variational inference with normalizing flows. In F. Bach and D. Blei, editors, Proceedings of the 32nd International Conference on Machine Learning. volume 37, Lille, France, 07-09 Jul 2015. PMLR. URL URLhttps::/proceedngg mlr.press. v37/rezende15 html.
[46] J. Richter, J. Nitzler, L. Pegolotti, K. Menon, J. Bichler, W. A. Wall, D. E. Schiavazzi, A. L. Marsden, and M. R. Pfaller. Bayesian windkessel calibration using optimized Od surrogate models, 2024. URL https: //axxiv www.iiv.rrg...e org/abs/2404 14187.
[47] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assister Intervention MICCAI 2015, pages 234-241, Cham, 2015. Springer International Publishing. doi:10.10077978-333199 24574-4
[48] Y. Ruckstuhl and T. Janjić. Combined State-Parameter Estimation with the LETKF for Sonvective-Scale Weather Forecasting. Monthly Weather Review, 148(4):1607-1628, 2020. doi: 100.1115/MWR---19.000
[49] P. Segers, E. Rietzschel, M. De Buyzere, N. Stergiopulos, N. Westerhof, L.. Van Bortel, T. Gille- bert, and P. Verdonck. Three-and four-element windkessel models: assessment of their fitting performance in a large cohort of healthy middle-aged individuals. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine, 227(4):417-428, 2008. doi: 10.100.24//05444 19JEIM287.
[50] Seidman, G. Kissas, P. Perdikaris, and G. Pappas. NOMAD: Nonlinear manifold decoders for operator learning. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances Neural Information Processing Systems, volume 35. Curran As- sociates, Inc., 2022. URL
http://pppppr........p.... cc hwwppppprreeeeeeeeeeeeeeeeeeeeee files / httpppppp/2////eeeee/ 24f 24429922881166111111984f66cco Abstract- Conference html.
[51] J. H. Seidman, G. Kissas, G. J. Pappas, and P. Perdikaris. Variational autoencoding neural operators. In Proceedings of the 40th International Conference on Machine Learning, volume PMLR, 23-29 Jul 2023. URL https: //tp::://rrroeee http///poceeennnn///ee mlr. www.mme....emeeeeeemmemm.m.eee press/v202.. hte//sidmmma dmma23a. html.
[52] S. Sherwin, L. Formaggia, J. Peiro, and V. Franke. Computational modelling of ID blood flow with variable mechanical properties and its application to the simulation of wave propagation in the human arterial system. International Journal for Numerical Methods in Fluids, 43(6-7): 673-700, 2003, 10. 1e..10000//d......
[53] Y. Shi, A. F. Gao, Z. E. Ross, and K. Arizzadenesheli. Universal functional regression with neural operator flows. In Advances in Neural Information Processing Systems: Workshop on Bayesian poccision-making and Uncertainty, 2024. URL https: /ttp:://oppeevieeeeeeeeee wtt.w/iwww...e..reeeeee net/forum? Id=ZCtnWuaZiI
[54] J. Sohl-Dickstein E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, Lille. France, 07- 09 Jul 2015. PMLR. URL. https: https::///roceeediggg pttppppioceenn..c// .mlr, html.
www.m.ll...///t/at..mm.mmmmm.a.t prwps/vesvv http//ohlldickcee
[55] R. Sokolow, G. Kissas, C. Beeche, S. Swago, E. W. Thompson, M. Viswanadha, Chirinos, S. Damrauer, P. Perdakaris, D. J. Rader, and W. R. Witschey. An aortic hemodynamic fingerprint reduced order modeling analysis reveals traits associated with vascular disease medical biobank. bioRxiv, 2024. doi:
14

---

[56] M. Straka, R. B. Wilhelmson, L. J. Wicker, J. R. Anderson, and K. K. Droegemeier. Numerical solutions of non-linear density current: A benchmark solution and comparisons. International Journal for Numerical Methods in Fluids, 17(1):1-22, 1993. doi: 10. 10.0000/fd. 16501 70103.
[57] G. G. Tong, C. A. Sing Long, and D. E. Schiavazzi. Vn networks: A data-driven framework for model synthesis and identifiability analysis. Computer Methods in Applied Mechanics and Engineering, 423:1 16846, 2024. doihhtps://doi..rg//... 101.:0.........ma22222.... 16846.
[58] R. Wang, Z. Chen, Q. Luo, and F. Wang. A conditional denoising diffusion probabilistic model for radio interferometric image reconstruction. In Proceedings of the 1 11th International Conference on Artificial Intelligence and Applications (FAIA), 2023. d6::10.3233FAAAA200554
[59] A. Wehenkel, J. Behrmann, A. C. Miller, G. Sapiro, O. Sener, M. C. Cameto, and J.-H. Jacobsen. Simulation- based inference for cardiovascular models. In NeurIPS Workshop, 2024. URL
https://arxiv. org/ abs/2307 13918.
[60] Wildberger, M. Dax, S. Buchholz, S. Green, H. Macke, and Schölkopf. Flow matching for scalable simulation-based inference. In Advances in Neural Information Processing Systems, volume 36. Curran Associates, Inc., 2023, URL https::/ nips cc wc/paper__feeeeeeeeeeeeeeeeeeeee _flles/
html.
[61] J. Xiong, X. Cai, and J. Li. Clustered active-subspace based local gaussian process emulator for ligh-dimensional and complex computer models. Journal of Computational Physics, 450, 2022. doi: 10: jop. 10840.
[62] J. Zhang, S. Zhang, and G. Lin. PAGP: A physics-assisted gaussian process framework with active learning for forward and inverse problems of partial differential equations, 2022. URL /ttp://arxiv org/ abs//204.00 02583.
15

---

A Supplementary Material
A.1 Training Details
All experiments were performed Nvidia GeForse RTX 3090 GPU with 24GB of memory. The number of training, validation, and test samples are provided in Table 3, along with the data dimensions. Further details regarding the experimental data are provided in Sections A.3 and A.4.
Training Strategy For each experiment, we performed a hyperparameter sweep when possible to select the learning rate, scheduler rate, weight decay, and network size. Due the large number of hyperparameters, we selected 64 random combinations of hyperparameters and trained each model for 500 epochs. Following the sweep, we selected the model with the best performance and trained it for 2000 epochs, saving it at the epoch with the best performance on the validation set. Following this, we evaluated its performance on the test samples. A special exception must be made for the VAE-based models, In VArt and e cVANO. In VAErt was particularly susceptible to posterior collapse. Following the advice of the authors in Section 2.3.2 of [57], we hand-tuned the model to prevent over- parreeerratttiitoooonn and used early stopping. Similarly, cVANO was susceptible to training instability and exploding gradients, which also necessitated careful tuning and training in order to get the best results. The final size of each model, in number of network parameters, is provided in Table 4. The results from the model selection revealed all models are fairly consistent in size, typically all on the same order of magnitude.
We observed some sensitivity to the normalization scheme used on the data. In the PWP experiment, we normalize each input channel to range between zero and one by dividing by the maximum value across all samples that channel. In the ACB experiment, we normalize each channel by taking the minimum and maximum value across all samples in that channel and rescaling these to range between zero and one.
The training time per epoch of all experiments is consistent, ranging from 1.5 to seconds. During the hyperparameter sweep, each model is trained for 500 epochs, taking approximately 12.5 minutes. The best model from the sweep is trained for a total of 2000 epochs, taking approximately 1 hour. Throughout this entire project, we estimate the total number of GPU-hours for all model hyper- parameter sweeps, hand-tuning, and training of the final model to be approximately 180 GPU-hours.
Additional Training Approaches Although the two objectives of FUSE are decoupled during training time, it is also possible to backpropagate losses the outputs through both the forward and inverse model components. FMPE supports differentiable sampling, therefore the loss on the outputs could be formulated as function of continuous inputs u.
La( (0; { p² = [ (o*(()))) G(p(5" (w)) [n)
La du.
(9)
In the case where the inputs and outputs, respectively u and s, are identical, a reconstruction loss may be applied to further train the inverse model component. This entails predicting outputs from the data-generating parameters 5- and predicting the inverse problem from the predictions s, as opposed to the original inputs u. This is formulated as
(o, 0) = (2(1; â = T (("())) CPMPE( (b: u = To G
(10)
Although each approach has the potential to aid training it is still required to train each objective separately. Likewise, we did observe improvements in our experiments by implementing these additional losses. This further supports our claim in Equation 1, that model components trained by decoupled objectives for forward and inverse problems may be unified evaluation to propagate uncertainties in the parameters to outputs and provide interpretable results for complex and dynamic systems of PDEs.
A.2 Evaluation Metries
Continuous Ranked Probability Score We define the Continuous Ranked Probability Score (CRPS) and present how compares single ground truth value y to a Cumulative Distribution Function (CDF) F, quantifying how well distribution of samples fits single target. The CRPS is
16

---

Table Data details. Ntrain Neat and Ntrain provide the number of training, validation, and test samples. The number of scalar parameters of the PDE system is given as m. "Channels" describe the number continuous input components (location/vaariables), and the number of points in the time discretization of the continuous inputs and outputs is given by "Sample Points". The "Batch Size" is the number of training samples per mini-batch during training.
Experiment
N Ntrain
Neal
Ntest
m
Channels
Sample Points
Batch Size
PWP
4000
128
128
52
487
64
ACB
8000
1000
1000
20
181
64
Table 4: Number of network parameters (millions) per model for the experiments.
Model
PWP
ACB
FUSE (ours)
7.2
5.8
UNet
14.1
9.8
cVANO
5.7
1.8
GAROM
11.1
2.7
VAEt
13.6
2.9
Table 5: Inference times and DDPM ablation. Inference time, CRPS, and relative L1 errors continuous outputs for trained model are evaluated as the number of FMPE and Conditional DDPM samples increase. Conditional DDPM implemented described [30] for 1D problem using 32 diffusion steps. We observe that the FMPE method results to higher accuracy in both metrics and slight increases in accuracy up until 5 512 samples, while DDPM is fairly consistent. Sampling times are approximately equivalent between the two models.
Posterior Samples
Forw. Pass
Inverse Pass (s)
CRPSx100
Erro
FNO
FMPE
DDPM
PMPE
DDPM
EMPE
DDPM
128
0.0018
0.179
0.108
1.63
0.58
0.58
0.91
256
0.0030
0.194
0.114
1.64
1 1.63
4.75
2.22
0.58
0.58
1.60 0.90
case: ACB
512
0.0059
0.211
0.114
1.34
1.34
1.64
4.61
2.16
0.58 = 0.58
1.60 0.900
1024
0.0127
0.240
0.113
1.49
- 1.64
4.57
1.722220
0.58 = 0.58
1.59 - 0.89
2048
0.0239
0.495
0.152
1.46
1.64
4.54
1.221
0.58 1 0.58
1.59 0.89
4096
0.0608
0.973
0.285
1.64
4.53
0.58 0.58
4 0.90
8192
0.1205
1.820
0.550
1.44
1.64
2.20
0.58 0.58
1.59 0.89
Table Evaluating training time, CRPS, and relative I.1 errors for models as the number of training data increases. As more training samples are used, CRPS and relative L1 error improve; however, the model converges after several thousand training samples resulting in smaller performance gains per new training sample.
Training Data
Training Time Per Epoch (s)
CRPS 10
L1 Error
128
0.027
16.10 4.53
5.70 2.34
256
0.059
12.61 + 4.85
4.22 2.29
case: ACB
512
0.118
10.23 4.31
2.51
1024
0.213
8.48
1.71
2048
0,450
5.88
1.93
4096
0,973
3.90
0.84
8192
1.698
1.84
0.58 0.60
defined as
CRPS(F.v) =E[ vll
[ X1
(11)
2
17

---

with independent and identically distributed random variables X and X' following the distribution dictated by F. The first expectation computes the mean absolute error of the samples compared to the target value, and the second expectation penalizes tight distributions which are far from the true value. CRPS of zero indicates that the distribution perfectly fits the target sample by returning Dirac measure.
Error employ the relative Lt and L error for evaluating predictions time series data. This metric is calculated by
- - (g)
(12)
||((E)|
where (( is the true function described by the system of PDEs, and (E) is prediction produced by the network.
18

---

A.3 Pulse Wave Propagation: Reduced Order Model for Blood Flow
The reduced order Swwerr-Stkke model considers a system of discrete compliant tubes and arterial segments, connected at points [1]. The length of the vessels is considered much karger than the local curvature and the wave propagation happens on the axial direction. Therefore, their properties can be described using a Cartesian coordinate x [52]. The luminal cross-sectional area is defined as A(x, 1) = SA da, the average velocity defined as (x,, X Sg u(x, t) ndo and the volume flux given cross section as Q(x, t) = UA A [1]. The arteries are allowed to deform in the radial direction due to the internal pressure P(z, t) which is considered constant over a cross section and the wall is considered impermeable. The blood is considered as an incompressible Newtonian fluid, with viscosity = 2.5m s and density p 1060kg m [13]. All the gravitational effects are ignored assuming a patient in a supine position [1]. Using the above assumptions, the pulse wave propagation is formulated as the following system of hyperbolic constitutive laws [1, 13]:
0A
OAU
0,
δt dr
(13)
0U
(2a 1)U OU
U² 04
10P f p 8x
dr (a-1)
+
Ot
A
8x
PA
1 Atra LA do the Coriolis coefficient that accounts for the non-linearity of the integration over the cross section and describes the shape of the velocity profile. The velocity profile commonly considered as axisymmetrie, constant, and satisfying the non-slip boundary conditions on the arterial walls. An example of such profile is [1]:
where t)
u(2, r, = - (
(14)
R
where is the radial coordinate, R(x, f) the lumen radius and Ç --1 constant. A flat profile is defined by 1 [7]. Integrating the three-dimensional Navier-Stokes equations for the axi- symmetric flow provides the friction force per unit length f(x, () = 2( + 2) To account for the fluid-structure interaction of the problem, pressure area relation is derived by considering the tube law. For a thin isotropic homogenous and incompressible vessel that deforms axisymmetricallyy at each circular cross section the following Voigt-type visco-elastic law [6] is typically used:
T(x) 0A
P Pa(A;3))
An(x) V A Øt
Pa(A, x) = Pex Ao( (VA - V An(x))
((x)
(15)
4 -(x)=3 (()
T(x) 2 vTb(a)h(())
where Pe is the elastic pressure component, h(x) the wall thickness, E(x) the Young's modulus, ø(z) the wall viscosity, 3(x) wall stiffness coefficient, and r(x) the wall viscosity. The scalars Pozs and An correspond to the external pressure and equilibrium cross-sectional area, respectively. The parameters ((r) and ((x) are computed using empirical relations found in the literature [7]. For this problem, a pulse wave from the heart is considered as a boundary condition for the inlet of the artery. For merging and splitting of arteries, the conservation of mass and the continuity of dynamic and kinematic pressure are considered, as well as no energy losses. For the outlet of the arterial system, boundary conditions that model the downstream circulation are considered. In [7], the three-element Windkessel model [49] is used:
Ri )+ + Q(1
OQ 0t
P. P
Pe
CR
(16)
R2
R₂
In [7], the authors consider algebraic relations from the literature that allow them to include different parameters, e.g. the volume flux, in the pulse wave propagation problem. For example, the digital PPG measurements can be related to the volume flux at the extremities as:
PPG(t) - 1.' Qw (t) Qour(t)
(17)
where T is the duration of a cardiac cycle, Qw the flow entering the Windkessel model, and Qout the flow in the outlet.
19

---

20
Table Cardiovascular parameters used in the reduced order model for the blood flow pulse wave propagation.
Parameter
Units
Description
Age
years
Age of the individual
HR
bpm
Heart Rate
SV
ml
Stroke Volume
CO
I/min
Cardiac Output
LVET
Left Ventricular Ejection Time
dp/dt
mmHg/s
Rate of pressure change in the heart
PFT
Pulse Transit Time
RFV
ml
Residual Filling Volume
SBP,
mmHg
Systolic Blood Pressure (arterial)
DBP.
mmHg
Diastolic Blood Pressure (arterial)
MAP &
mmHg
Mean Arterial Pressure
PP
mmHg
Pulse Pressure (arterial)
SBP,
mmHg
Systolic Blood Pressure (brachial)
DBP&
mmHg
Diastolic Blood Pressure (brachial)
MBP,
mmHg
Mean Blood Pressure (brachial)
PP,
mmHg
Pulse Pressure (brachial) Pulse Pressure Amplification Augmentation Pressure Augmentation Index
PP PPamp
ratio
AP
mmHg
AIx
Tr
ms
Reflection Time
PWV.
m/s
Pulse Wave Velocity (arterial)
PWVef cf
m/s
Pulse Wave Velocity (carotid- (erooiddfemoral)
PWV br
m/s
Pulse Wave Velocity bbrachial-radial
PWV fa
m/s
Pulse Wave Velocity cemmoral-nnkle)
diaasea
mm
Diameter of Ascending Aorta
diaato
Diameter of Descending Thoracic Aorta
diagbda
mm
Diameter of Abdominal Aorta
diagar
Diameter of Carotid Artery
Len
mm
Length
drop
mmHg
Pressure Drop at Finger
dropankte
mmHg
Pressure Drop at Ankle
SVR
10 Pa / m
Systemic Vascular Resistance
Table 8: Artery locations used in the reduced order model for the blood flow pulse wave propagation.
Artery Location
Description
Aortic Root
The portion of the aorta that is attached to the heart Major arteries in the neck leading to the brain, neck, and face The part of the aorta that runs through the chest The major artery of the upper arm Artery located in the forearm, commonly used to measure the pulse The part of the aorta that runs through the abdomen
Carotid
Thoracic Aorta
Brachial
Radial
Abdominal Aorta
Iliac Bifurcation
The point where the aorta splits into the common iliac arteries Arteries that supply the pelvic organs and lower limbs
Common Iliac
Femoral
The major artery supplying blood to the thigh and lower limbs Artery located in the lower leg
Anterior Tibial
Sup. Middle Cerebral
Artery that supplies blood to the cerebral hemispheres
Sup. Temporal
Artery that supplies blood to parts the face and scalp
Digital
Arteries that supply blood to the fingers

---

PWV.
300
14
1
(c) Information level 3
14
Figure A. PWP. inverse problem: Histograms for the sample with the median CRPS for each level of available input information.
De
125
PAY
16
Lo
369
PWW.
16
MINE
11
dayk
(b) Information level 2
DRI
sanl
2WY.
1
dawe
S
10
240
Si 20.
RPY
PP.
96
PPT
MAP.
(a) Information level
DBP
XP
26
500
1WW..
Lm
21

---

PW
PWN
1t
JAC
Figure A.2: PWP. inverse problem: Histograms for the sample with the greatest CRPS (worst case) for each level of available inpur information.
PWW
1
(c) Information level 3
15
1B1
PWY
Jana
PWY.
Loa
300
PWW.
200
10
PWW.
11
Its
(b) Information level 2
180
DWI
PWW
100
PWY.
11
Lon
180
11
poW.
197
PP2.
20
PWV.
11
Information level
opss
PWW
11
PWV.
km
100
22

---

Figure A.3: PWP, propagated uncertainty: Pressure time series for the sample with the median rel. C1 Error for each level of available input information.
(c) Information level 3
24
(b) Information level 2
1 70mg
P Paweer
(hmi (1 Mad
(a) Information level
23

---

Figure A.4: PWP, propagated uncertainty: Pressure time series for the sample with the greatest rel. L1 Erro (worst case) each level of available input information.
(c) Information level 3
(b) Information level 2
Farmyy 1
5
8
de
H de
66
H
H
02
45
(a) Information level
M 1
05
4 O4 1
02
15
61
Plame Sereen
24

---

25
0.125
15
Rolasioo Biroo
CntS
10
0.075
0.050
0.025
Test Case
Test Case 2
Test Case 3
(a) CRPS on the inverse problem.
(h) error on the forward and unified problem.
Figure A.5: PWP, performance summary: Box plots of the errors the forward and inverse components of FUSE, as reported in Table 1, for different levels of available input information ("test cases"). For the inverse problem, parameter samples č; are sampled from o (5*1lu) problem, the output function is predicted based on the true parameter values 5 ~ p(glu). For the unified problem evaluating both the inverse and forward model parts, the mean & of the ensemble prediction s; from inferred parameters & (E u) is compared to the true output time series As information removed from the input the different cases, becomes more difficult estimate s.
the forward
0.35
0.30
0.25
Relative L1
0.20
0.15
0.10
0.05
Aortic
Carotid
Thoracic
Brachial
Radial
Abdomiual Aorta
Common
Femoral
Anterior
Sup. Middle
Digital
Rool
Aorta
Bifurcation
Ilae
flaac
Tibial
Cerebral
Temporal
Figure A.6: PWP, propagated uncertainty: Relative L Error per vessel predicting pressure time series from the true parameters.

---

26
Figure A.7: PWP, forward problem: Predictions with the largest L emoo using the true parameters of the cardiovascu lar model.
Coptar Sax
Sac Teaprnal
04
00
00
100
100
130
105
110
Sig S05 Drekel
6.
C2
C2
3
3
Pijentt canmen)
Figure A.8: PWP, inverse problem: In the case of fully masked input (all waveforms are zero). FUSE (blue) returns distributions which closely match the training (prior) distri- butions (orange).
200
100
300
10
PPc
PPA
T:
200
600
0.00
0100
40
040
25.3
SV
16.0
0.00
0.00
0.00
010
010
90
DRRP,
G1
010
8000
00.0
02-
05-
690 01
500
S33
60
310
05
03

---

27
Figure A.9: PWP. inverse problem: In Vn predictions over parameters. the median sample with full input information (level 1). In VAEt is particularly susceptible collapse, i.e., the predictions over the parameters become overconfident and the uncertainty bounds fail to capture the true parameter.
posterior
I0e 300 PWY.
T0PV
2P.
30
to MAP.
pWW.
12
F3R
dppt
rep.
DRR.
dnggge
PWN.
16
55
Presuue [sauate]
Presure foe
Avetic Root
100
100
75
0.6
Time (+)
Time (6)
(a) Age: 25 (blue) to 65 (red)
(b) DBP: 55 (blue) to 90 (red)
Fresare jumnti)]
Presure (mmnie
125
Avetie Root
Aurtic Root
100
100
70
Time (9)
Time (x)
(c) HR: 60 (blue) 90 (red)
SBP: 80 (blue) to 165 (red)
Presuue jamnttk
125
Pressure [mmn
Acette Root
Aurtic Root
100
0.2
0.4
0.6
Time
Time
(6)
LVET: (blue) to 360 (red)
(I) MAP: 75 (blue) to (red)
Figure A010: PWP, sensitivity analysis (fingerprint): Pressure in the Aortic Root. We study how the pressure changes due to parameters of the system. Blue represents the lower bound of possible values, and red represents the upper bound, with colors of varying hue at intermediate values.

---

28
17.5
Case
Case
15.0
Case
12.5
Relative L1 Error
10.0
7.5
5.0
0.0
0.02
0.04
0.06
0.08
0.10
0.12
0.14
CRPS
Correlation of the errors between function outputs (L') and finite- dimensionalparrmeetrrs (fR different levels of available input information ("levels", or "cases").
Figure A.11:
PWP:

---

A.4 Atmospheric Cold Bubble
In atmospheric modelling, large-eddy simulations (LES) help to understand the turbulent dynamics within convective storms [39] that are usually observed only by scalar local measurements of wind (anemometers), other scalar measurements obtained at surface-level weather stations (e.g.. pressure, humidity), and, if available, three-dimensional wind lidar records. A particularly destructive phenomenon associated with storms are downbursts, sudden local downdrafts that cause extreme winds when hitting the surface. Manual tuning of a numerical model is usually required to fit simulation to match an observed downburst (c.g.. [40]), due to the model complexity involving several parameterizations such as cloud microphysics. The model can then be configured to explore the sensitivity of the extreme winds both to the numerical model parameters and to the physical environmental conditions [38]. In addition to improving the scientific understanding and forecasts for more precise warnings, simulations of downbursts help to design robust infrastructure. such as wind turbines. However, the amount of simulations performed for model tuning (inverse problem) and sensitivity analysis (forward problem) is limited by the high computational cost to typically tens to hundreds of simulations. The affordable number of numerical simulations is either used for strategic expert tuning or to train an emulator for the parameter-to-die evaluate and can then be used for in-depth sensitivity analysis (e.g., Gaussian Processes [9]: quadratic regression [36]). We would like to highlight that the presented test case is conceptually similar to downbursts, but far from simulating real-world scenarios due to its idealized nature and reduced complexity. However, we believe that test cases like this one will help to pave the way for handling realistic cases with methodologies such as FUSE.
map, which is cheaper to
Model Description The numerical data for the cold bubble simulation is obtained with the PyCLES model [42]. It solves the anelastic Navier-Stokes equations as formulated by [41], evolving around a hydrostatic reference state,
8po OX0 Des
g,
(18)
with reference profiles ao(z) for specific volume and (2)2) for pressure, and (I1, 22 xx) = (I,, v) and (u1, U2, Uz) (u, u, w) for simplicity. The anelastic equations of motion are given by
Ou;
1 (pou,us)
Doop'
1 ((poTts))
+ +6113
+ Ei,
(19a)
01
Po
Ox;
Dx.
Po
Ouj
Ös
1 (pouss)
Q
1 ((p)aa) + 5,
(19b)
Øt.
Po
Om,
T
Po
0x.
Opous
0,
(19c)
Ox;
encompassing the momentum equations (a), the entropy equation (b), and the continuity equation (c). We use standard conventions for summing and the Kronecker delta dez. and left out the terms and further equations referring to the Coriolis force and moisture, as they do not apply to the small-scale and dry cold bubble simulation. The buoyancy term depends on the specific volume, moisture, gravity, and the reference pressure profile po. p' is the dynamic pressure perturbation over the hydrostatic reference state, and Σ, is an additional momentum source term, which is expected to be small for the cold bubble experiment. In the entropy equation, is the diabatic heating rate, T is the temperature, and S is source term of irreversible entropy sources. Finally, Tay and Yo, the sub-grid scale (SGS) stresses acting on velocity and entropy. which model the effect of turbulence smaller than the grid scale in a diffusive way as
Tij Sig.
(20)
relating the sub-grid stress to the strain rate S,j = 1/2(8.u, + +,uu)) of the resolved flow. The eddy viscosity v may be related to the diffusivity D. for heat and other scalars Φi, and ultimately to the corresponding SGS stresses To.i by the turbulent Prandtl number Pr, by
Dr Ta.i = - -D, dz:
//Pr
(21)
For dimensionl simulations, one may choose a sub-grid scale parameterization for is such as a Smagorinsky-tyee first-order closure or a higher-order closure based on prognostic equation for
29

---

Table ACB:
Discrete parameters with their ranges used for uniform sampling of the training data, as well as default values used by [56]. The parameters either encode the initial condition (IC) or are part of the sub-grid scale (SGS) model (M). The horizontal location &i kept fixed to ensure the horizontal symmetry of the domain.
Parameter
Name
Unit
Type
min
default
max
horizontal location
km
IC
26.2
Ir
horizontal radius
km
IC
Ze
vertical location
km
IC
2.5
3
3.5
Zr
vertical radius
km
IC
2
2.5
amplitude
K
IC
5
15
25
V/
eddy viscosity
m²/s
M
75
75
D2
eddy diffusivity m²/s
M
75
75
turbulent kinetic energy. For the two-dimensional cold bubble case, however, stick to uniform values for both Vi and D,, independent of the turbulent Prandtl number.
The domain of size (Ir,, I,2) (51.6 km, 6.4 km) exhibits neutrally stratified background profile (0_0- O, where / denotes potential temperature), and no background winds. elliptic cold air anomaly is prescribed by
AT
-a (cos + /2, [|(( - x,)=,,, (= zc) 2z)2,
(22)
where default values and ranges for all parameters can found Table 9, with the resulting flow fields shown in Figure A.12.
The PyCLES model by [42] relies on a WENO finite volumes scheme that exhibits exceptional stability, implemented in Cython. It is used here in the setup described by the authors, at resolution Ar A 50 m and an adaptive time stepping bound to At S 5 order to record time series measurements temporal resolution of five seconds.
Numerical Setup
The data is recorded the eight locations marked in Figure A.12, horizontal locations 15, 20 km (about 10 and 5 km from the center of the anomaly). and vertical heights of 50, 100, 250, 500, and 2000 m. each being part of the numerical grid.
In terms of computational complexity, a PYCLES simulation in the given setup takes about half an hour eight cores.
30

---

4.0
1-0
4.0-
1-0
4.0
5m1
1-4.8
6-
11.0
[0.0
5:000
4.0-
5:900
t= 9.00
2.0
2.0
2.0
10.00
10.0
15.0
20,0
55.0
20.0
15.0
20.3
25.0
10.0
15.0
20.0
25.0
0-00
4.0
4.0
2.0
0.0
0.0
1.0
L=100
4,0
I=100
4.0
L=
2,0
2.0
0.0
0.0
10.0
15.0
20.0
35.0
10.0
15.0
20.0
25.0
10.0
15.0
20.0
35.0
[Kau
23
10
20
10
10
m//
anmusly [K]
(m/
Figure A. 12: ACB: Time evolution of horizontal velocity (left), temperature anomaly (middle), and vertical velocity (right), for the default values of the parameters given Table showing three main vortices that develop until T = 900 s. Measurement locations for the time series data are marked with triangles, with their location indices indicated in the first row.
1-00
4.0
5-00
3
2.0
10
2.0
10
2.0
1:0
6.0
1-4
6.6
000
10.0
15.0
20,0
25.0
10.0
15.0
20.3
25.0
10.0
15.0
20.0
25.0
$00.0
4.0
9.0.0
4.0
900.0
0
2.0
100
[00
15.0
L. 020.0
4,0
L. 02100
4.0
2.0
2.0
2.0
11.0
10.0
15.0
20,0
25.0
10.0
15.0
20.3
23.0
10.0
15.0
20.0
25.0
100.0
1.0
90000
1.0
L000
2.0
8.0
110
0.0
15.0
[kmi
[am)
mn/4]
comperature anmodly [K]
[m//s]
Figure A.13: ACB: As Figure A.1 for sample with stronger anomaly, reaching the state of three main vortices around t = 600 s instead of t 900 s.

---

Figure A.14: ACB, inverse problem: Samples drawn from the parameter distributions given continuous time series data. The true parameter values are marked with dashed lines, while the predicted distributions are given in blue shading.
pol0/=")
99.7%
95.6%
68.3%
1.6
(b) Parameter estimates. from the test sample with median CRPS.
3.6
3.0
7.
6.0
3.5
3/
16
60
diffusivity
16
5.0
3.6
1 3.0
2.4
2.4
1.6
99.7% %
95.3 1
Pat0
(a) Parameter estimates from the test sample with largest (worst) CRPS.
08.3
1.6
3.6
3.0
80
5.0
3.5
16
diffusivity
30
5.0
18 3.0
2.4
2.4
1.6
32

---

(c) Sample with strong velocity gradients (Fig. A.13).
1900
400
400
400
600
000
100
Loodine
Locaiin
Lockking
Figure 15: ACB, forward problem: Velocity predictions with the FUSE model using the true parameters for different samples from the test set.
200
200
100
900
100
600
600
00
40
Lookuee
Leckke
Lockoee 5
400
300
300
200
300
1900
160
400
400
660
600
000
(b) Sample with median of FUSE.
Locoties
Locktoe
Locdlns
260
20
26
100
600
600
400
600
199
Looaeen 0
Locokin S
Lookeee
200
100
100
(a) Sample with (wors)) FUSE.
Vid
mi
Eoeration
Thet 2n
The 34t
Locetton
Locadeen
20
20
140
900
600
400
40
Dookene
33

---

34
diffusivity
viscosity
amplitude
40
40
20
20
20
20
25
50
75
25
50
75
10
20
2.5
5.0
7.5
2.5 3.0
3.5
1
2
(a) Sample which FUSE has the median CRPS.
diffusivity
viscosity
amplitude
Zr
10
40
20
25
25
50
20
2.5
7.5
2.5 3.0
3.5
(b) Sample on which FUSE has worst CRPS.
Figure A.16: ACB, inverse problem: Histograms of the parameters inferred from continuous time series measurements u, for different test samples. Given the very small amplitude and horizontal extend of the worst-case sample, this perturbation hardly reaches the sensor and the parameters hence difficult to infer from the weak velocities measured.
Les (Dikas, Mika)
Loc. (ilee, Sixe
Loc. (1Sk,, Slwe
1oc (D3e,,
(a) Jr
(b) Amplitude
(10ke,
Zr
(d) Ze
Lex. (1lle. Mas)
Lec. (18e 93a0
3: (15kk
Loc. (12kka, 200)
-10
-15
Viscosity
(f) Diffusivity
Figure A.17: ACB, sensitivity analysis: Fingerprints showing the model sensitivity to one parameter time location I, while keeping all others their default value.

---

35
350
-113
30.0
+0.10
150
130
100
10.0
30
1100
500
1.00
2.0
5.0
30
25.0
20.0
20.0
15.0
30
150
10.9
180
H5
25.
30.0
80
30.0
25.0
25.0
20.0
20.0
150
29
2.
Ailairis
(a) Location (z
15 km,
50 m)
(b) Location 5 = 20 km, 50
Figure A.18: ACB, sensitivity analysis, continuation of Fig. 4: Validation of the FUSE model against numeric simulations on peak horizontal velocities at location 1 (left, further away from the perturbation center) and 5 (right, closer). From top to bottom: relative error between FUSE and the numerical model, difference between FUSE and the numerical model. maximum velocity calculated by the numerical model, maximum velocity calculated by FUSE.

---

36
(a) FUSE model.
(b) Numerical ground truth.
Figure A919: ACB, sensitivity analysis: Peak horizontal velocities u location (x km, 2 = 50 m), sampled for pairwise combinations of the parameters, while keeping all others at their default value. For the neural model (a), 100 samples are drawn for each parameter pair, corresponding to 150,000 evaluations. For the numerical model (b), four simulations were run per pair of parameters, with each taking values corresponding to 1/6 and 5/6 of the parameter range, corresponding to 60 model evaluations.
(a) FUSE model.
(b) Numerical ground truth.
Figure A.20: ACB, forward problem: Same Figure A.19, location 5 (x 20 km, 2000 m).

---

37
Relative Loss
1.5
1.0
0.5
(1ökm, 100m) u
1
(1ökm,, 500m) u
1
(15km, 2km) u
W
(20km, 500m) u
1
(15km, 50m)
3: (16km, 270m)
(20km,, 50m)
(20kmn, 100m)
(20km. 250m)
10: (20km, 2km))
(a) Forward problem: Error between the prediction on the true parameters {" and the true output time series 8.
2.5
2.0
Relative Li Loss
1.5
1.0
0.5
0.0
I: (15km, 50m)
2: (15km, 100m)
3: (15km, 250m)
(15km, 500m)
(15km. 2km)
(2tkm,, 50m)
(20km, 100m)
(20km, 250m)
9: (20km, 500m)
10: (20km, 2km)
(b) Unified prediction: Error between the predicted ensemble mean based the input time series u, and the true output time series s.
Figure A.21: ACB: Box plots of relative errors in the time series predictions each location.

---

A.5 Properties of the Total Variation Metric
Consider the total variation distance d(u, n²), where j and jr are measures over a E U, which is defined as
(() sup (m(A) -²" ACU
=sup/ ACUl I u(uudu I(x 10 (u)du).
(23)
Further, consider a random vector E E C R", the function h E - U, and the probability measures p and p" over E. Since pushforward measures induced by p through the function h, μ = h and μ = hy We assume all measures admit densities and use the same notation for measures μu( A), C Ut and their densities (()), и E U4 since the distinction is clear from the arguments.
h(E), the measures and μr" be considered the
We consider the conditional probability measures p(Clu), p² (Elu). Then, and μ² are given by:
u(u)= / ((E))(Elu)dE, p²(u) = p² P'(s)
and the measure set A is:
m(A) / pl p(u)du- J (())
1² / (0(u)du- S (Elu)dddd
-
Assume u uniquely determined by through the map u = l(()) and that p(Elu) hence highly concentrated at h (u) and therefore:
p(Elu) = 8(E - h -u))
p² (glu) 8(5 h h- (u))
where ó is the Dirac delta function and
m(u)= = [ p(E)O(E - 11 I = p(h~1(m))
1r" (w) [ (' - (w))
('
The definition of the total variation is then written as:
p(h (w))dd - p² p'
d(,,p)) = sup
ACU
A
lp(h ((()+((A)) - -(((h)(A)))
(24)
sup
ACU
d(p, p*).
Therefore due the unique definition of u from via h, the definitions of the total variation cquivalent. The relation between the total variation of the original and the conditional measures is then given by:
dp,pp)) = d(p,p') < / d(((E)) =
(25)
38

---

A.6 Detailed Derivation of the Forward Objective
We start with d(S) #y: 94y) and our goal is to bound this metric by the supervised operator learning objective. For function u, its corresponding images Ç(u) and Go (u), and for any set E B(S), where B is Borel o-lleebra over S:
(#r(A)= - w ((()))GHH(()) ( (a = ( (8),
where ws(s) is the indicator function that is one if s E A and zero otherwise. Let B' (2) be the a-algebra on UI generated by the pre-images of sets in (SS) under G. Then,
/ wa(s) ( - wa(s)do (())) ( ILe w (x(((u))du()) (
d(g (au) =
sup AEB(S)
(
sup
1 ((wx(((w)) w (900 (U)
(26)
sup
< J.. (ā(u) - -
< [ ll((w) - (w)
Here, I| ())+ is the L norm over the space of function S. We are interested in parametric PDEs where have access parameters ę E = C Rm, the PDE parameters, sampled from p. probability measure on Ik"". Operator learning involves a map between continuous functions, so an intermediate step needs be considered to first lift the parameters & to a continuous space before performing the action of the operator. For this purpose, we can, for example, consider h to be the band-limite lifting described in the main text, composed using lifting ( 1 - Bg and an inverse Fourier Transform F- BB - U. In this formulation, I maps the parameters to l to the Paley-Wiener space of band-limited functions in Fourier space, B₁ {Z E [ supp c [-2,, [2], thus truncating the frequencies that can be used to represent the data. We consider μ p(t()), the pushforward of p({) under h. and use a change of variables:
[ f(u)dp(u)= f(h(E))dp(E),
for any integrable function f. We can apply this property to:
1 (wx(((()) - w (u)((duuu sup ÀEB(U))
d(G G2p)
/ ((=)(((M(E)) w (0,
sup AEB(())
We can define the operators
g 3: S and : E S and write:
((wx(0(E) W (O
d(G
22 = sup
(EB(S)
< / |[ (()))
Finally, we arrive at
d(g ( - [g((E) - (E))))l
(27)
39

---

A.7 Detailed Derivation of the Inverse Objective
In the supervised operator learning problem, we have access to samples u of functions but we do not have access the distribution that generates u. Assume that 9 is Lipschitz continuous:
|| ( (u))l I
Assuming that ja is an approximation of js, we can relate these two measures by the total variation of their respective pushforward measures under the forward surrogate G using the continuity properties of the operator (see Appendix A.5),
Myo ( S C d( .1),
for some constant C E I, which we generally assume to be equal to one. This property holds because of the continuity of Ço and because the total variation is preserved under continuous transformations. We assume that the functions u are generated by solving a parametric PDE, where the parametric uncertainty over & is propagated onto u through the likelihood μ(ul(). Performing parameter estimation corresponds to sampling from a conditional distribution (Elu)) approximating the posterior p(E u). To do so, first bound the previous distance by the total variation between p (Elu) and ((lu) over E (see Appendix A.5):
d(pe, 1)) (()))
(28)
Combining the two inequalities, we get
d(O G S (E)u), which means that we can minimize the right hand side of the inequality (29).
(29)
A.8 Formulation of the FMPE loss
For our implementation of Flow-Matching Posterior Estimation (FMPE), we closely follow [60]. In the main text (Eqn. 5), we formulated the FMPE objective as
d(p (Elu), p(E)u)),
for the total variation distance d on measures the parameter space = R". We will minimize this distance in order to approximate the underlying distribution, p == p. and sample M samples from p (E)u) with the help of FMPE. We also defined the transform (6).
= T (u) à h 1 pds (u).
which transforms an dimensioal input E l/ to finite-dimensional conditional information E TRR in Fourier space.
The FMPE method learns an invertible transform between the target distribution of parameters, conditional on the additional information, and standard normal distribution. This transform is modelled as an ordinary differential equation with artificial time dimension E [0, I| and solution
v/ Rm p
d dt vv.a(()) = ve, (V), vo.a = Eo.
(30)
where Eo follows a standard normal distribution. Trajectories in the artificial time can be obtained as & = (E). This way. the target probability ()) is approximated as
(Elú) (vi,a) - (O( (- 1 div us (E)(())
by solving the transport equation d. div(p. = 0.
During training, vo (Ex) is fit to resemble a standard normal on the training samples {Et, ú, ) During evaluation, standard normal sample En is drawn, and transformed to sample that approxi- mately follows p(E (() as
E' ~1,4, (Eo) p(Elu).
(31)
In order to fully describe a.. it hence suffices to learn the trajectory velocity function V2.a- Let (ElEE) ) be sample-conditional conditional Gaussian probability path with vector field 4, (E+15),, and let time be distributed as t p(t). Then, [60] formulate the FMPE loss as
02 CFMPE (6) = Beeotttssc-oddt///--dddeeede-- (
(Ex) - (..11))
(32)
40

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The claims made in this paper are clearly highlighted in the abstract and introduction. The main contributions have been listed clearly in the introduction through bullet points. The results section is structured such that justifies these claims.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes] Critical assumptions and practical limitations are presented in the Discussion
Justification:
section.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: All assumptions and formulas clearly stated, numbered, and cross- referenced the paper with theoretical justification and detailed derivation provided in the appendix.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
The experimental setup is described extensively, with model architectures and training details outlined in the appendix. Furthermore, we provide all code, data, and models (including baselines) required to reproduce the experiments outlined in this paper.
Justification:
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
The data and code is provided in the supplementary material, uploaded with the paper submission.
Justification:
6. Experimental Botting/Detail
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Training details described for the all experiments, with more extensive training details specified in the appendix and Supplementary Material. Additionally, all the selected model configurations provided in the code.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
41

---

Justification: Errors or scores are reported by the meant/-standard deviation across all test samples. Furthermore, figures displaying box plots of these errors and scores are presented in the appendix. The quantities that these numbers and figures represent are clearly stated in the respective text and captions.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The amount of compute required for each the experiments, as well as the total amount of compute hours for all experiments, is outlined in the training details reported in the appendix.
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: htttp:://neurppstt http://uurriss.ccppbbbepe wwtppblic//eteeeeeeeeeeeeeeeeeee htt.ae//ehhhsssieennnnneeeeeeeee
Answer: [Yes]
Justification: This work complies with the NeulPS code of ethics, with special attention to the data and model documentation, access research artifacts, and disclosure of essential elements for reproducibility.
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
This work is primarily focused on a broader strategy solve problems in scientific computing, i.e. solve systems of differential equations. Direct paths to negative applications. such deepfakes or generation of misinformation are not present. Additionally, while this work presents experiments on synthetic cardiovascular data, it does not meet the NeurlPS guidelines immediately cause negative societal impact without a prior evaluation protocol. For health data specifically, detailed evaluation protocols issued by public health and safety organizations must be considered to meet in-silico, in-vitro, and in-vivo evaluation standards.
Justification:
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
The data used in these experiments has a very low risk for misuse. The dataset which released with this work is based on fluid dynamics for turbulent systems, while the other dataset used in experiments is publicly available, synthetic dataset of cardiovascular models.
Justification:
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The dataset for full-body haemodynamics is open source and appropriate cited and credited. The atmospheric turbulence dataset is produced by the authors, but based on an openly available code cited in the text. Baseline models used for comparison are properly cited in the text.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
42

---

Answer: [Yes]
Justification: The code that reproduces the results is documented and easy to read. The documentation is contained in the actual code.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects. 15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
43


Reviews:
[{"review_id": "MeJtFOSnSk", "reviewer": "Reviewer_X7vi", "summary": "The paper introduces a novel framework called FUSE that unifies surrogate modeling and parameter identification for parametric partial differential equations (PDEs). Traditionally, field prediction and statistical estimation of discrete parameters have been separately handled by using operator learning surrogates and simulation-based inference, respectively. FUSE proposes a combined approach that aims to enhance the accuracy and robustness of both tasks.\n\nFUSE is designed to jointly predict continuous fields and infer the distributions of discrete parameters, leveraging a unified pre-training step to amortize the computational costs associated with both the inverse and surrogate models. It employs a probabilistic extension of operator learning through Flow Matching Posterior Estimation (FMPE) to effectively handle parametric uncertainties. This unified approach facilitates in-depth model calibration and supports sensitivity analysis and uncertainty quantification (UQ) at a significantly reduced computational cost.\n\nThe authors demonstrate the proposed methodology on two applications: pulse wave propagation (PWP) in the human arterial network and an atmospheric cold bubble (ACB) simulation. They emphasize the advantages of FUSE in both the inverse (parameter estimation) and forward (continuous field prediction) tasks.", "questions": "1. In the supplementary section (Fig A.1, A.2), it seems that the identified parameters can cover a very wide range. However, these parameters do not significantly impact the PDE solution (Fig A.3, A.4). Is it correct to interpret these variables as having low sensitivity?\n\n2. In the worst case of (Fig A.3, A.4), even when considering uncertainty (i.e., various possible parameter values), the obtained solution seems quite different from the true one. Should this be interpreted as indicating the presence of unknown physics? \n\nThese questions might arise from my lack of familiarity with the specific applications discussed, as mentioned in the weaknesses above.\n\n3. Can the methodology in this paper be applied if the boundary conditions are set as parameters?", "limitations": "The authors have effectively explained the limitations of their approach in the paper.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. Problem Definition: The authors have mathematically defined the statistical estimation of discrete parameters, a significant problem encountered in practical applications.\n\n2. Integrated Approach: FUSE provides a holistic and efficient solution for addressing parametric uncertainties in PDEs by integrating surrogate modeling and parameter inference tasks within a single framework.\n\n3. Robustness: The use of FMPE in a probabilistic setting makes the model enhance the robustness of model predictions under varying conditions and uncertainties.", "weaknesses": "1. Lack of Novelty: This paper appears to be an application of FMPE (Flow Matching Posterior Estimation) to the inverse problem, lacking significant novel developments.\n\n2. Uncommon Problems: The problems addressed in this paper have not been widely discussed within the machine learning community (though it might be due to my lack of familiarity).\nQuestions", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would first like to thank the reviewer for acknowledging the novel framework, the rigorous problem definition, and the integrated approach we take constructing a robust framework.\n\n**W1**\nRegarding the concern with respect to the lack of novelty in the FMPE application, we would like to highlight key differences and contributions in this work.\n\n1. **This is not merely an application of FMPE:** The metric in the first term in Eq. (1) is bounded by a metric as shown in line 127. Because this metric is hard to compute in practice, our implementation matches $\\rho$ and $\\rho^*$ with FMPE; however, it may be substituted by other choices such as NPE or diffusion models (see 1-page pdf). In other words, FMPE is not essential to the methodology and any suitable model for matching posterior distributions would suffice.\n\n2. **Inverse Problem:** Additionally, this paper introduces function space simulation based inference, to infer parameters from functions as opposed to vectors. Therefore, we provide an inverse model that is grid resolution-invariant, bridging a gap between neural operator architectures and simulation-based inference. This is a critical concept for many practical applications and, to the best of our knowledge, has not yet appeared in the literature.\n\n3. **Forward Problem:** We present a formulation of the supervised operator learning problem for a map between a finite-dimensional parameter and an infinite-dimensional space of output functions. This is accomplished by a novel Lifting Operator, transforming finite-dimensional vectors into the space of band-limited functions, as presented in lines 115-116.\n\n4. **Unification:** We propose a novel framework in the operator learning literature that unifies forward model simulation and statistical parameter estimation under the same rigorous mathematical framework in Eq. (1). The joint formulation of both problems using the triangle inequality allows to assess what we refer to as the propagated uncertainty. This approach enables a more comprehensive understanding within many scientific problems by introducing explainable model uncertainties, which tie uncertainties in complex, infinite-dimensional spaces to uncertainties in simple parameters which characterize the system.\n\n**W2**\n**Uncommon problems:** We recognize that the two problems presented may not be widely known to the broader ML community; however, ML-based model calibration is of vital interest in many applied communities. In particular, cardiovascular models have been subject to both classical and ML-based inversion due to their relevance [arXiv 2307.13918]. Climate science is also a major topic within the field of operator learning (arxiv: 2208.05419, 2111.13587, 2306.03838). Due to their high complexity, there is less work on the calibration of small-scale atmospheric LES models, even though their parametric uncertainties are well-known [doi 10.1007/s10546-020-00556-3]. However, the combination of numerical methods and ML presents an opportunity to greatly improve the state of the art [paper ref. 48].\n\n**Q**\nTo answer the questions, let us first clarify that for the Pulse Wave Propagation problem, we present three scenarios with different levels of input information. In cases 2 and 3, most of the measurement locations on the human body and quantities of interest are not available, and hence masked (lines 196-200). When the reviewer refers to the \"worst case\", we assume they mean case 3 with least information provided.\n\n**Q1-2.** First, there are indeed some variables with low sensitivity to the parameters (e.g. Fig. A.9: pressure is insensitive to age), which generally have wider posterior distributions. The wider distributions in Fig. A.1-2.b-c are a result of larger uncertainties as the number of input channels is reduced from 39 (case 1) to 3 (case 2) and 1 (case 3). It would be reasonable to interpret parameters with large uncertainty as a result of low sensitivity to the limited set of inputs in these cases. The uncertainty arising from the ill-posed nature of these cases, as opposed to unknown physics, results in pressure predictions which are quite different from the true values. Essentially, the parameters with a strong sensitivity to pressure may not be determined because they have a weak sensitivity to PPG at the fingertip (case 3). The result is that a more \"average\" pressure is predicted, with large uncertainty ranges.\n\n**Q3.** Yes, in fact, the Pulse Wave Propagation experiment has boundary conditions encoded as parameters. The left boundary conditions (i.e. the pulse wave from the heart) is parameterized by Heart Rate, Stroke Volume, Pulse Transit Time, Residual Filling Volume, and Left Ventricular Ejection Time. We kindly refer the reviewer to see original reference [35] for more detailed information.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5amrY0rGIi", "reviewer": "Reviewer_JJ2R", "summary": "The authors propose \"FUSE\", a combination of multiple neural operator models, which are trained to jointly solve PDE forward problems and perform parameter inference of given parametric PDE.\nThe main idea is to start from a range of PDE solutions obtained from various parameter values, and then train neural operators that can interpolate both in parameter space and in solution space.\nThe approach is evaluated on two systems of parametric PDE, and compared to a range of similar neural operator and sampling approaches.", "questions": "1) l64: why is the framework called \"FUSE\"? Is it an acronym for something?\n\n2) l93: should it not be $\\mu$ instead of $\\mu^*$ on the right argument of d as well? Otherwise, how can we minimize the distance to the full measure, not just its approximation?\n\n3) l142: what does it mean that \"$\\mathcal{P}$ is a map that lifts the channels of the dimensions of the input function\"? What does \"lifting channels of dimensions\" mean?\n\n4) The authors state that (l31) \"iterative and thus expensive calibration procedures\" are required for classical solvers as a drawback, but then 180 GPU hours (l604) are required for training the proposed model. The authors do not comment on this, so my question: why is it beneficial to train, or \"calibrate\" the neural operator model for such a long time, as opposed to using the \"iterative and expensive\" calibration procedures for classical solvers?\n\n5) There are very little details on the \"learnable maps\" mentioned throughout the explanation of the approach. Which maps are used? Are all of these MLPs, deep, which nonlinearities? These details do not need to be mentioned in the main paper, but even the appendix does not contain them.\n\n6) There is very little detail on the the applicability of the approach. For which PDEs is it useful, when will it not work, how much data is needed for which types of problems, etc.? These questions do not need to be answered completely, but there is not even a simple example where this is being discussed or studied.", "limitations": "The authors discuss limitations in terms of future work, which is fine. Negative societal impact is not discussed, where the authors argue that only PDE problems are being solved, without immediate concerns - which is questionable, especially because the authors use pulse propagation in the human cardiovascular system as an example where the approach could be used (and where humans may be harmed it it does not work in practice in a hospital).", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The results of the computational experiments seem impressive, especially given that only a relatively small number of samples is used (O(1000)). The main idea of jointly learning the parameter inference and the forward PDE solution is valid and interesting. The comparison to a range of other approaches shows that the proposed approach works well.", "weaknesses": "1) Figure 1 is not very clear. The caption does not explain what is shown in the figure, the parameters, models, inputs and outputs are not mentioned.\n\n2) Many references are only referred to by their pre-print. The proper journal / conference should be given instead.\n\n3) Inference times are not provided, and no comparison to classical solvers is performed (even though they are being used to generate the training data). Also see question 4.\n\n4) The paper (main and appendix) does not contain a lot of details regarding implementation and applicability (Questions 5 and 6).\n\n\nMinor:\n1) l88: \"observatbles\" -> observables\n2) l110: G is now not an operator as defined in l87, but a function on a finite-dimensional space (with a range in a function space). The redefinition is confusing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1.**  We thank the reviewer for pointing this out. We include an updated Figure in the one page document, which we believe is much more explanatory. \n\n**W2.**  Even though referencing pre-prints is a common practice in ML, the reviewer is right in that, if available, the respective journal or conference of publication should be provided. We will update all references accordingly.\n\n**W3.**  The inference times for FUSE are in the order of milliseconds per sample, details are provided in the one page document. As for the classical solvers, the computational complexity for the ACB case is about half an hour on eight cores per sample (lines 709-710). As cited in line 192, the PWP data set was not created by the authors, but by (Ref. [35]). The authors do not report the computational time, but simulating 1 sample of the system using the OpenBF open source Julia code [openbf-hub, A. Melis, 2018; Melis 2018, EthosID uk.bl.ethos.731549], takes about 80 seconds on one core. As the reviewer points out, this comparison is crucial to justify the use of ML models, and will be emphasized in future versions of the paper.\n\n**W4.**  See Q5 and Q6.\n\n**Minor weaknesses.**  We thank the reviewer for pointing out the typo, which we will of course correct. Regarding the notation, however, we would like to emphasize that l110 defines $\\mathcal{G}$, while line 87 defines $\\tilde{\\mathcal{G}}$. We chose this notation to distinguish the unified operator $\\mathcal{G}: \\mathcal{U} \\mapsto \\mathcal{S}$ from the parametric forward model $\\tilde{\\mathcal{G}}: \\Xi \\mapsto \\mathcal{S}$.\n\n**Q1.**  FUSE stands for Fast Unified Simulation and Estimation, as shown in the title. To facilitate the reading, we will repeat the full name where pointed out by the reviewer.\n\n**Q2.**  We thank the reviewer for asking for clarification on this notation, which we are convinced is correct. The mathematical problem formulation merely presents the distance $d$ that corresponds to the problem on a theoretical level, involving both the true unknown $\\tilde{\\mathcal{G}}$ and $\\mu^*$. Following common practice in ML research, $d$ is subsequently upper bound by loss functions involving $\\tilde{\\mathcal{G}}$ and $\\mu^*$, which are approximated by finite sampling when it comes to implementation, as suggested by the reviewer.\n\n**Q3.**  The \\emph{lifting function} is a commonly used term in ML research, also heavily used in the operator learning literature [3, 9, 10, 31], that describes an affine (linear) transform that increases the dimensionality of a function $u \\in \\mathcal{U} \\subset \\mathcal{C} (\\mathcal{X}, \\mathbb{R}^{d_u})$  defined as $\\mathcal{P} : \\mathcal{C} (\\mathcal{X}, \\mathbb{R}^{d_u}) \\rightarrow \\mathcal{C} (\\mathcal{X}, \\mathbb{R}^{\\tilde{d}_u})$, where $\\tilde{d}_u >> d_u$. In practice, a lifting function is implemented by a one-layer fully connected neural network.\n\n**Q4.**  As discussed in lines 601-604, 180 GPU hours is the total computational time for 64 runs of hyperparameter sweep for all models and all experiments contained in the paper, while training a FUSE model only takes about 1 GPU hour. Accounting for the wall-clock times of one simulation of ACB and PWP problems (W3), it is infeasible to repeatedly calibrate the numerical solvers using traditional methods for varying function input $u$ (e.g., MCMC requires about 1,000 to 10,000 simulations for each $u$). However, once trained, the low inference times of the FUSE model (W3) and its operator properties make this calibration possible.\n\n**Q5.** \"Learnable map\" denotes a parametric function (for instance a neural network or an affine map) whose parameters are learnt during training. We followed standard practice in ML and did not define this terminology, while believing that an interested reader can check the code for exact architecture of our maps. However, we agree with the reviewer that defining these maps is more reader-friendly and will do so in a camera-ready version, if accepted.\n\n**Q6.**  As explained in the introduction, the method proposed in this paper is applicable to any parametric PDE. It will not work for PDEs that are not a priori parametric, e.g. for a stochastic PDE with a Brownian motion forcing, as explained in the limitations section. An estimate of the sample complexity for different PDEs would require a statistical analysis on a range of problems, which is outside of the scope of this paper. However, we provide an analysis of error scaling with the number of samples for the ACB case in the 1-page pdf. We observe that even if we consider 4096 training samples, half of what we used originally, FUSE would still be more accurate than any of the baselines reported in the main text.\n\n**L.**  The reviewer raises a point about the reliability of the FUSE model if it were applied in clinical practice for the Pulse Wave Propagation case. We believe it is very commendable of the reviewer to think about potential negative societal impact of this method. \n\nNegative Societal Impact, as per NeurIPS author guidelines, is considered for cases where the presented methodology can be immediately used without a prior evaluation protocol, e.g., deep fakes. We present a general purpose framework and do not think that FUSE can be used as a clinical decision making tool as is. For this to happen, a very detailed protocol issued by a public safety organization needs to be considered that contains multiple steps of in-silico, in-vitro and in-vivo validation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "24k7CoBZBt", "reviewer": "Reviewer_s1Yg", "summary": "The authors study the problem of joint prediction of continuous fields and statistical estimation of parameters for physical systems governed by PDEs.  Prior work had focused on operator learning and then inference to determine the statistical parameters. Here, the propose to solve for both jointly in their method FUSE which combines Neural Operators with Flow Matching Posterior Estimation (FMPE). The authors then test their method on important applied problems in haemodynamics and large eddy simulations showing advantages in both the inverse and surrogate problems.", "questions": "1, In parametric PDEs, are the authors discussing the PDE parameter or the BC/IC as mentioned in the introduction and why are they discrete? The authors should clarify this.\n2. How are the two loss functions weighted and counter-balanced? Is there a hyper-parameter to tune?", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Good application domains studied including haemodyanmics and atmospheric large-eddy simulation of bubbles.\n- Interesting problem to study parametric PDEs\n- Identified proper limitation of numerical methods when the PDE parameters are not known exactly and calibration techniques must be used to learn the parameter from data in inverse problems.\n- Nice overview of Neural Operators\n- Nice to also mention ROMs and recent work in deep learning.\n- Good that the paper tackles both forward problems with UQ and inverse problems\n- CRPS is good uncertainty metric\n- Also good that OOD case is tested\n- Thorough an diverse evaluation\n- Results show that the proposed FUSE method is performing strongly\n- FUSE maintains the discretization invariance property of NOs", "weaknesses": "- Add references to the classical numerical methods, such as LeVeque in the introduction\n- Can add reference to the Multiwavelet Neural Operator, Gupta et al., NeurIPS 2021 in the introduction.\n- Missing literature references: Neural Operator methods with UQ for the Forward Model Simulation including Bayesian Neural Operator, Magnani et al., and see the overview and detailed comparison in Mouli et al., \"Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs\", ICML 2024, which studies sensitivity analysis and UQ for Neural Operators. I think these works should be cited and the methods compared to as baselines. Also see Ma et al., \"Calibrated Uncertainty Quantification for Operator Learning via Conformal Prediction\", https://arxiv.org/abs/2402.01960, 2024 for conformal prediction techniques.\n- The main limitation is lack of benchmarking against the above FNO + UQ baselines\n- More recent state-of-the-art baselines, e.g., diffusion models (DDPM, DDIM) and variants could also be compared.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1.**  The reviewer rightly points out that a reference and comparison to classical numerical methods is indispensable when justifying the use of ML-based methods. Since the application field \"parametric PDEs\" for our method is rather broad, we are happy to adopt the suggestion to reference a standard text book on the numerical solution of PDEs, as provided by LeVeque in 1992.\n\n**W2.**  We thank the reviewer for highlighting the multiwavelet neural operator (MWT), which is an interesting addition to our extensive review of neural operator approaches in lines 43/44. Since it proves superior performance to FNO in the given reference, it would be an interesting future study to replace the FNO by MWT within FUSE, in particular when it comes to test cases with larger scale separation.\n\n**W3, W4.**  In order to avoid any confusion on terminology up front, we would also like to point out that when Magnani et al. 2022 refer to operator learning for \"parametric PDEs\", the parameters taken as input to the learning problem are functions in space, $\\lambda(x)$. Our setting, in contrast, assumes the parameters $\\xi$ to be vector-valued constants, as an intermediate step between the functions $u$ and $s$. We assume that when suggesting to use uncertainty quantification (UQ) for neural operators (NO), they refer to the forward model part $\\Xi \\mapsto \\mathcal{S}$ only.\n\nWe thank the reviewer for the suggestion to add the references on UQ for NOs. However, we respectfully disagree on the applicability of UQ for NOs to the problems considered in our paper. Let us explain why: UQ for NOs, in the sense indicated by the given references, quantifies the approximation error by the NO model, instead of the parametric uncertainty in the physical model targeted by FUSE. Thus, benchmarking against the proposed methods, which is listed as the main limitation of our paper, is not possible. As Mouli et al. 2024 point out in their review in the appendix, it is common practice, e.g. in weather forecasting, to perturb a physical model input and parameters to quantify the uncertainty related to the physical state and the model itself, respectively. We deem it important to distinguish these types of uncertainty from the additional approximation error when fitting an ML model. They are also easily distinguished by the probabilistic mappings they model: While an ensemble approach maps an ensemble of parameters (as provided by the FMPE) onto an ensemble of predictions (pushforward), the UQ approaches for NOs equip a prediction on a single input with an uncertainty range. In order to clearly interpret the uncertainties given by FUSE as parametric physical model uncertainty, and given the good performance of the model in our id and ood validation, we consider the PDE parameters as the main source of uncertainty.\n\n**W5.** The reviewer raises an excellent point.  Flow matching and diffusion are competitive methods each with their own advantages and disadvantages. As explained in \"Flow Matching for Generative Modeling\" (ArXiv 2210.02747), flow matching may use a diffusion defined probability path, but an optimal-transport path may also be selected, as in FMPE. The result is that FMPE is able to train on less data, converge faster, as well as provide faster sampling at inference time. Given that PDE training data often come from expensive numerical simulations, data efficiency is a key component for machine learning approaches in scientific computing. To showcase this, we have run experiments using a conditional DDPM model and present the results in the 1-page pdf. Even though this is initial work, we observe that the preliminary experimental findings support the above arguments. We will include this discussion and results in future versions of the paper.\n\n**Q1.**  We thank the reviewer for the question, which is crucial to the applicability of our method. Indeed, the \"parameters\" can stem from model properties (ACB: $v$ and $d$, Table 7), or parameterizations of the initial (the remaining ACB parameters) or also boundary conditions (PWP: pulse wave from the heart, lines 644-646  and original reference), or any other model component that is parameterized.\n\nConcerning the nomenclature \"discrete\", we acknowledge that it is misleading, and we will change it to \"finite-dimensional\" parameters $\\xi$, as opposed to the infinite-dimensional $u(t)$ and $s(t)$. We mean to point out that the inferred parameters are vector-valued constants that are not space or time-dependent.\n\n**Q2.**  The two loss functions are fully decoupled, see line 153 of the experiments section. So, there is no need for hyperparameter tuning.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T0ac3w7BNN", "reviewer": "Reviewer_goJy", "summary": "This paper proposes a framework to tackle simultaneously forward (simulation of the system) and inverse (estimation of key parameters of the system) problems for PDEs. Namely, the authors suppose the existence of an underlying parameter $\\xi$ that characterizes the input functions $u$ of the PDE, and therefore formulate a probabilistic framework where the output solution $s$ is sampled according to $\\xi \\sim p(\\xi | u)$ and $\\hat{s} = G^\\theta(\\xi)$. The authors employ a two-loss objective for training where the first one aims to train the operator and the second one is used to approximate the true distribution $p^*(\\xi | u)$. The method is tested on complex PDE systems, that depend on multiple parameters, namyely atmospheric cold bubble (ACB) and plus wave propagation (PWP). It obtains better performance than existing methods and convincing uncertainty propagation results.", "questions": "* Do you have access to the true parameters $\\xi$ during training ?\n* Do you train the neural operators with parameters $\\xi$ sampled from the flow matching ?\n* Is the training done sequentially or in parallel ? According to the minimization objective, each loss only depends on a single set of parameters.\n* How many steps do you take at inference ?\n* Did you try other generative models other than flow matching ?\n* Did you try a fully probabilistic framework ? For instance assuming that the observations of $s$ were not deterministic given $\\xi$ ?\n* What is the training and inference time of the method ?", "limitations": "There is a limitation section that discusses the assumptions of the method.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper proposes FUSE, a unifying approach for both inverse and forward problems for PDEs, with a relevant theoretical framework.\n* The method obtains SOTA results on complex PDEs on the forward metrics, and is very competitive in the inverse task. It outperforms all the baselines in the OOD regime, showcasing the robustness of the method.\n* The uncertainty propagation property is impressive, as shown in Figure 4 and 5.\n* They propose the first application of flow matching for PDE problems, and integrate Fourier layers in the architecture.", "weaknesses": "* The datasets used are quite complex as they include many parameters and several equations. It could be best to provide additional visualizations of the data and further describe the data format used by each block of the architecture to better illustrate the different components.\n* The paper does not justify the use of flow matching compared to a different probabilistic approach. Would there be a difference between DDPM and Flow matching in this case ?\n* There is a lack of details on flow matching, particularly on the parametrization $\\psi_{u, t}$ and its inverse $\\psi^{-1}_{u, t}$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1.**  We thank the reviewer for pointing out possible difficulties in understanding our model and data. An updated version of the model illustration is provided in the 1-page pdf, including the requested details. We are happy to incorporate any further specific suggestions, should we have missed a crucial part. In terms of data visualization, we point to the extensive figures in the appendix, in particular the sensitivity studies in Figs. A.9 and A.16, as well as A.11 showing the data-generating 2D fields for the ACB experiment.\n\n**W2.** The reviewer raises an excellent point. Flow matching and diffusion are competitive methods, each with their own advantages and disadvantages. As explained in \"Flow Matching for Generative Modeling\" (ArXiv 2210.02747), flow matching may use a diffusion defined probability path, but an optimal-transport path may also be selected, as in FMPE. The result is that FMPE is able to train on less data, converge faster, and provide faster sampling at inference time. Given that PDE training data often come from expensive numerical simulations, data efficiency is a key component for machine learning approaches in scientific computing. To showcase this, we have performed some initial  experiments using a conditional DDPM model, and present the results in the 1-page pdf. Keeping in mind that these are only preliminary results, we observe that the experimental findings support the arguments stated above. \n\n**W3.**  We understand the reviewer's concern and agree that more details regarding FMPE models would benefit this paper. However, given the amount of space available, fully describing these details was not possible within the main text. To clarify, We have followed [23] in this regard (lines 135-144), and we would kindly refer interested readers to their work for a full derivation and explanation of flow matching for simulation-based inference. We would also include details on FMPE in the appendix of a CRV, if accepted. \n\n**Q1, Q2.**  The true parameters are used during training, with training samples being tuples of the form $(u,\\xi^*,s)$. The inverse part of the model is trained with $(u,\\xi^*)$ to learn a $u$-conditional probability path to $\\xi^*$, while the forward map is learned from $(\\xi^*,s)$. Only at evaluation time, when merely $u$ is available as input, the parameters $\\xi$ sampled from the flow matching are input to the forward neural operator to obtain $s$.\n\n**Q3.**  Training is done in parallel. As the reviewer correctly points out, each loss function only depends on a single set of parameters during training. At evaluation time, we combine these two models to unify the inverse and forward problems, enabling us to study the relationships within the system, such as uncertainty propagation and sensitivity analysis.\n\n**Q4.**  We assume the reviewer is asking about the number of artificial time steps known from diffusion models.\nAlthough FMPE is defined along a time-dependent probability path, this pseudo-time variable is not discretized as in a diffusion model, but computed in a single step. \nShould the reviewer be referring to the number of samples drawn by the FMPE, our results use 100 samples from $\\rho(\\xi | u)$ at inference time. \n\n**Q5.**  After considering other generative approaches in the initial phase of our project, such as discrete normalizing flows and generative-adversarial models, the flow matching approach presented in this work proved the most competitive in our research. Please refer to W2 for a comparison to diffusion models.\n\n**Q6.**  In our work, we did also investigate a fully probabilistic approach. However, given that these data sets come from numerical simulators of PDEs whose continuous outputs are fully deterministic given the input parameters, fully probabilistic models in the way proposed by the reviewer offer no advantage in capturing the parametric uncertainty. As the reviewer suggests, this fully probabilistic approach would be interesting to investigate for instances where the observations are not deterministic given the parameters, such as with SDEs. We leave this interesting line of research to future work.\n\n**Q7.**  We provide the training and inference times in the 1-page pdf, and thank the reviewer for requesting this important information.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2L1aswYlY7", "reviewer": "Reviewer_4oLp", "summary": "The authors propose a new approach, FUSE to the simultaneous learning of an emulator and statistical estimation of underlying \"discrete\" parameters in a joint training step.   \n\nThe approach splits the problem into two:   (i) the forward problem which is modelled through a FNO neural operator approach,  effectively learns a map from a space of finite-dimensional parameters to an output function.   (ii) The inverse step then seeks to learn a conditional distribution for the parameters $\\xi$ based on measurements of the continuous input $u$ using a flow-matching approach.    This yields two loss functions which are simultaneously optimised.\n\nThe authors present two challenging problems as test cases.", "questions": "The limitations indicated above need to be clarified -- some clarity is needed in the introduction to spell out the applicability of this methodology.   If my understanding is correct, then stronger motivation of why a combined approach is even needed.\n\nDiscrete parameters is not really the right terminology -- which seems to suggest your data is ordinal valued (1,2,3).   I suspect you mean that your parameter is a finite dimensional vector rather than a function?\n\nFinally, the literature review seems very focused on neural network approaches, yet there are other approaches based on Gaussian processes etc.  Could the authors provide some relevant background which is more encompassing.", "limitations": "Limitations have been well addressed.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The authors have identified an important problem which is genuinely challenging and have proposed a potential solution to this problem.   They have demonstrated the effectiveness of their approach on two challenging PDE-based problems.   The components of the proposed approach are not novel, but their combination seems to be a new approach, demonstrating originality.", "weaknesses": "I struggled at times to understand the actual methodology in the paper.   The two strategies for forward and inverse are relatively clear, but their combination is far from clear to me,  which leads me to question the broad applicability of this method.\n\nMy understanding of the methodology is that the authors require data of the form $(y, u, \\theta)$,  for which $u$ is a continuous input, $y$ is a continuous output and $\\theta$ is an intermediate set of parameters.     The forward model learns the deterministic map between $y$ and $\\theta$ and the inverse model learns the probabilistic relation between $\\theta$ and $u$.    If this is the case, then the need to propose a combined methodology seems unclear to me.     The fact that the two optimisation problems in (4) appear to be decoupled, suggests that this is the case.\n\nThe challenge that the authors describe is most relevant in settings where data from $\\theta$ or $u$ are not available, in which case we must rely on $y$ and $u$ to identify $\\theta$ while training the model,  but this seems to be not the case?  Otherwise, I am struggling to see the need for this combination of methods.\n\nI also think the authors could be more comprehensive in their literature review, which has been very narrowly focused on a handful of methods.  There are both more recent methods, and quite a history of older approaches which seek to address this problem in different contexts around science and engineering.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1.** **Concerns about the novelty/applicability:** We thank the reviewer for this important question. We will exemplify the applicability of the methodology by providing an example in PWP. The goal in this setting is to predict the output function $s$ given a finite-dimensional vector of parameters $\\xi^*$. In problems such as personalized medicine, $\\xi^*$ is not known a priori, as it represents quantities such as vascular resistance that cannot be measured non-invasively, and it needs to be inferred from available measurements $u$, e.g. PPG. This is a strongly ill-posed problem, thus a probabilistic approach needs to be considered. This is modelled by approximating a posterior probability $\\rho(\\xi^*|u)$. When the posterior probability is approximated, posterior samples are drawn and used to run a flow solver to estimate continuous quantities such as local vascular pressure $s$, which cannot be measured non-invasively, and uncertainty estimates. More generally, FUSE targets the problem of calibrating vector-valued parameters of PDEs, which is applicable to many areas in the sciences such as climate modeling, computational fluid dynamics, material science, wave scattering etc. \n\n**W2.** **Reasons for Unification under a Neural Operator framework:** We understand the reviewer's confusion and thank them for asking for clarification. There are two strategies that can be considered for the inverse problem, which for the sake of brevity we are going to call \"classical\" and \"ML-based\". Given measurements $u$ the \"classical\", (e.g. MCMC) strategies guess the value of $\\xi^*$, and iteratively correct their guess. For complex problems such as the ones presented here, these approaches are computationally infeasible due to the high cost of the numerical solver, e.g. see lines 709-710 for ACB, to compute the likelihood probability. \n\nThe \"ML-based\" approaches are built in two stages. They first consider a *training* stage where given pairs of $(u_i, \\xi^*_i), i=1,...,N$, with $u_i = $Sim $(\\xi^*_i)$ and Sim a numerical solver, the model learns an approximate distribution $\\rho(\\xi_i^* | u_i)$ using variational inference or normalizing flows. During the *evaluation* stage a new input $u$ is given and the ML model provides samples from $\\xi^* \\sim \\rho(\\xi_i^* | u_i)$. To get an ensemble of continuous quantities $s$ and uncertainty estimates, following both the \"classical\" and \"ML-based\" inversion strategies, a numerical solver needs to be considered, which is very expensive as well.\n\nFor this reason, we propose a methodology that uses the same ML architecture to infer $\\xi^*$ given $u$ and to act as a solver surrogate to make predictions of $s$ given $\\xi^*$ jointly during evaluation. During training we consider all $(u, \\xi^*, s)$ as known but during evaluation we consider only $u$ known and predict $(\\xi, s)$. Using a neural operator surrogate for the solver, we can consider $u$ to not be equal to $s$, and e.g. get predictions at only specified locations of interest, e.g. the aorta, without need to run the simulation through the whole cardiovascular system. \n\nThe joint analysis of the inverse and forward problems with FUSE results to the following important benefits:\n\n1. **Propagated parametric uncertainty:** The joint formulation of both problems under a rigorous mathematical framework (Eqn. 1) allows assessing the influence parametric uncertainty in the physical model has on the predicted continuous quantities in a very efficient manner.\n\n2. **Unified validation:** Since both the forward and inverse models are nonlinear it is hard to track the error accumulation in the different components. The joint evaluation is a solution to assessing forward and inverse errors separately. Furthermore, this mathematical framework facilitates the comparison of different combination of inverse/forward models and training objectives. A priori, the errors of an inverse and a forward model trained and validated separately may amplify nonlinearly at concatenation, increasing the chance that samples generated by the former are OOD for the latter.\n\n3. **Function space simulation-based inference:** Parametric PDEs as described above require a communication between the infinite-dimensional function spaces of $u$ and the finite-dimensional space of $\\xi$. In order to allow operating on function spaces, we combine a finite-dimensional flow matching model with neural operators, which is (to the best of our knowledge) an entirely new concept. This approach can generalize to any flow based or diffusion approach. \n\n\nAnswers to the questions:\n\n**Q1.**  We kindly refer to the answer provided above.\n\n**Q2.**  We thank the reviewer for pointing out this ambiguous nomenclature and will adopt the proposed change. The term \"discrete\" emerged in opposition to \"continuous\" spatially varying parameters, which is of course not the right wording.\n\n**Q3.**  We are happy to incorporate any literature the reviewer may be pointing at that amplifies our literature review in the proposed direction. Following the reviewer's suggestion, we would like to add several references based on Gaussian processes for forward and inverse problems (arxiv 2204.02583), kriging (2012.11857), and GP based models to explore complex high-dimensional spaces by low dimensional representations (2101.00057), as well as others.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
At the outset, we would like to thank all reviewers for their valuable time and feedback. We believe this discussion will lead to meaningful improvements in the quality and presentation of our work, improving its accessibility to practitioners of scientific machine learning. Furthermore, we would like to express our gratitude to the reviewers' recognition of our novel contributions to this field and the rigorous analysis of our approach.

As per guidelines, we are uploading a 1-page pdf with the following contents,

1. To address the comments of several reviewers, we have conducted initial experiments using a conditional denoising diffusion probabilistic model (DDPM) within the FUSE framework, in the place of FMPE. The results for both approaches, comparing inference time and accuracy for different numbers of posterior samples, are presented in the attached 1-page pdf. 

2. We provide details on the scaling and training times of the FUSE model employed in our experiments with respect to the number of training samples. 

3. We provide an updated main figure which we believe will add clarity to the types of data, general model framework, and training approach.


In the following, we will address the detailed comments of each reviewer in their respective rebuttal fields. We hope to address the concerns of all the reviewers with our detailed rebuttal and request them to kindly update their assessment. 

Yours sincerely

Authors of *FUSE: Fast Unified Simulation and Estimation for PDEs*

Meta-review: The reviewers unanimously recommend acceptance (6-5-6-7-6). The author-reviewer discussion has been constructive and has led to a number of clarifications and potential improvements to the paper. The authors are asked to implement the clarifications discussed with the reviewers in the final version of the paper.
Decision: Accept (poster)