Title: FUSE: Fast Unified Simulation and Estimation for PDEs
Venue: NeurIPS 2024
Accepted: yes
Keywords: Forward and Inverse Problems

PDEs

Neural Operators

Neural Posterior Estimation
Abstract: The joint prediction of continuous fields and statistical estimation of the underlying discrete parameters is a common problem for many physical systems, governed by PDEs. Hitherto, it has been separately addressed by employing operator learning surrogates for field prediction while using simulation-based inference (and its variants) for statistical parameter determination. Here, we argue that solving both problems within the same framework can lead to consistent gains in accuracy and robustness. To this end, we propose a novel and flexible formulation of the operator learning problem that jointly predicts continuous quantities and infers distributions of discrete parameters, thereby amortizing the cost of both the inverse and the surrogate models to a joint pre-training step. We present the capabilities of the proposed methodology for predicting continuous and discrete biomarkers in full-body haemodynamics simulations under different levels of missing information. We also consider a test case for atmospheric large-eddy simulation of a two-dimensional dry cold bubble, where we infer both continuous time-series and information about the system's conditions. We present comparisons against different baselines to showcase significantly increased accuracy in both the inverse and the surrogate tasks.

FUSE: Fast Unified Simulation and Estimation for PDEs

Abstract

The joint prediction of continuous fields and statistical estimation of the underlying discrete parameters is a common problem for many physical systems governed by PDEs. Hitherto, it has been separately addressed by employing operator learning surrogates for field prediction while using simulation-based inference (and its variants) for statistical parameter determination. Here, we argue that solving both problems within the same framework can lead to consistent gains in accuracy and robustness. To this end, we propose a novel and flexible formulation of the operator learning problem that allows jointly predicting infinite-dimensional quantities and inferring distributions of finite-dimensional parameters and thus amortizing the cost of both the inverse and the surrogate models to a joint training step. We present the capabilities of the proposed methodology for predicting continuous and discrete biomarkers in full-body haemodynamics simulations under different levels of missing information, and in a test case for atmospheric large-eddy simulation of a two-dimensional dry cold bubble, predicting continuous time-series measurements from inferred system conditions. For both cases, we present comparisons against different baselines to demonstrate significantly increased accuracy in both the inverse and surrogate tasks.

1 Introduction

Partial Differential Equations (PDEs) describe the propagation of system conditions for a very wide range of physical systems. Parametric PDEs consider different system conditions as well as an underlying solution operator characterized by a set of finite-dimensional parameters. Traditional numerical methods based on different discretization schemes such as Finite Differences, Finite Volumes, and Finite Elements have been developed along with fast and parallelizable implementations to tackle complex problems, such as atmospheric modeling and cardiovascular biomechanics. For parametric PDEs, these methods define maps from the underlying set of parameters, which describe the dynamics and the boundary/initial conditions and which we assume to be finite-dimensional here, to physical quantities such as velocity or pressure that are continuous in the spatio-temporal domain. Despite their successful application, there still exist well-known drawbacks of traditional solvers. To describe a particular physical phenomenon, PDE parameters and solvers need to be calibrated on precise conditions that are not known a priori and cannot easily be measured in realistic applications. Therefore, iterative and thus expensive calibration procedures are considered in the cases where the parameters and conditions are inferred from data [2]. Even after the solvers are calibrated, an

Dana Grund

Georgios Kissas

AI Center

ensemble of solutions needs to be generated to account for uncertainties in the model parameters or assess the sensitivity of the solution to different parameters, which are computationally prohibitive downstream tasks [43].

Related work A variety of deep learning algorithms have recently been proposed for scientific application to PDEs, broadly categorized into surrogate and inverse modeling algorithms, to either reduce the computational time of complex forward simulations or infer missing finite-dimensional information from data to calibrate a simulator to precise conditions.

Surrogate learning is a paradigm for accelerating computations. Hence, the cost of evaluating continuous quantities is amortized to an offline training stage. For functional data, the so-called Neural Operators [28, 3] generalizes to different conditions and discretizations of a complex system (resolution invariance). A multitude of operator learning approaches has been designed [37, 27, 50, 21, 44, 33, 32, 17], such as the Fourier Neural Operator (FNO) [31]. However, a priori, neural operators are not designed to process maps between finite-dimensional and continuous quantities. A different surrogate modeling strategy is constructed by learning a map between PDE parameters and solutions, as in traditional reduced-order model (ROM) approaches [16, 8, 19], and their deep learning counterparts [23, 10], including Generative Adversarial ROM (GAROM) [10]. In addition, Gaussian processes and kriging have been revisited in the context of PDE emulation [15, 61].

Amortizing the cost of parameter inference has also been widely explored in the literature of generative modeling and Simulation-Based Inference (SBI). In SBI, Invertible Neural Networks are combined with discrete Normalizing Flows to develop Neural Posterior Estimation [11, 12, 24, 45] or continuous Normalizing Flows in Flow Matching Posterior Estimation (FMPE) [60, 35]. However, they commonly rely on an expensive physical solver to sample continuous predictions, and are hindered by the use of approximate physical models, Gaussian distributions, or a limited sample size.

Approaches that attempt to jointly infer parameters and learn a surrogate are much rarer in the literature. Many are built on Variational Autoencoders (VAEs) [57], in finite [25] or infinite [51] dimensions. They inherently assume an underlying bijective relation between the functional and scalar quantities or are restricted to Gaussian latent representations, which limits their applicability to nonlinear problems. Among them, InVAErt [57] extends a deterministic encoder-decoder pair with a variational latent encoder. However, InV AErt is primarily constructed for investigating identifiability of parameters in systems of PDEs, and may not be suitable for learning uncertainty propagation from parameters to continuous fields. The conditional Variational Neural Operator (cVANO) [26] makes use of the resolution invariance of neural operators, based on [51]. cV ANO relies on a V AE to learn a low-dimensional manifold of the system constrained by finite-dimensional parameters, but is restricted to Gaussian approximations which are not suitable for problems which lack bijectivity. As alternative approaches, Simformer [14] estimates both finite-dimensional and function-valued parameters using transformers within the SBI framework, and PAGP [62] leverages the expressive power of Gaussian Processes. However, both are not suited for functional input due to quadratic growth with the input size. Finally, OpFlow [53] brings together operator learning and normalizing flows for quantifying uncertainty in output functions, but it lacks interpretability and inference on the latent space.

Our contribution Based on the experience and limitations of the aforementioned approaches, we formulate the following requirements for a unified forward-inverse framework relating functional quantities to a known space of finite-dimensional parameters: First, the interpretability provided by the given parameterization should be retained in both the forward and inverse tasks as well as their combination. Then, both the forward and inverse model should be discretization invariant with respect to all functional variables. Finally, a general probabilistic representation should be chosen for the latent parameter space, allowing for arbitrary parameter distributions. Following these guidelines, the main contributions of our paper are the following.

We propose Fast Unified Simulation and Estimation for PDEs (FUSE), a rigorous framework for unified inverse and forward problems for parametric PDEs. The FUSE framework, illustrated in Figure 1, allows to combine different forward and inverse models, and we choose to instantiate it with Fourier Neural Operators (FNOs) and Flow-Matching Posterior Estimation (FMPE) for the experiments in this work.

We formulate a mathematical framework with a unified objective for the forward and inverse problems, which allows us to assess how uncertainty in the inverse problem propagates

through the forward problem ( propagated uncertainty ) and evaluate both models together to avoid nonlinear error amplification at model concatenation.

We showcase how to adapt forward surrogate and inverse estimation models to fit the FUSE framework, mapping between finite and infinite-dimensional spaces. In particular, we extend FNO with a custom lifting to take finite-dimensional inputs, and we equip FMPE with an FNO-based encoding to allow for functional conditional information.

Our experiments show that our implementation of FUSE overcomes struggles of baselines (cVANO, GAROM, InVAErt) and an ablation (U-Net) on two complex and realistic PDE examples, pulse wave propagation (PWP) in the human arterial network and an atmospheric cold bubble (ACB). The experiments exemplify its ability for accurate and fast surrogate modeling, parameter inference, out-of-distribution generalization, and the flexibility to handle different levels of input information.

To our knowledge, the extensions we present to FNO and FMPE have not been explored in the literature yet. We would like to emphasize that we chose to base ourselves on these methods to illustrate the FUSE framework, but other forward and inverse methods may be more suitable for particular test cases.

n'wp-oo

fin.-dim. ∈ IRm

s'wp-oo

Inverse

Forward

Problem

Problem

mxM

dxM

Band

Inverse

Lifting +

Fourier

Posterior

Operator +

Limited

Fourier

n

Operator

Transform

Estimation

Lifting

Transform

Projection

S

*

Training:

(u, E*,s)

Evaluation: (u, ?, ?)

2 Methods

Notation and Assumptions Let U ⊂ C ( X, R d u ) and S ⊂ C ( Y, R d s ) be spaces of continuous functions on compact domains X ⊆ R d and Y ⊆ R d ′ , respectively, and let Ξ ⊆ R m be a space of finite-dimensional parameters. For a map F : A → B , where we take A and B to be among the spaces U , S , or Ξ , and a probability measure π ∈ Prob ( A ) , we denote by F # π ∈ Prob ( B ) the push-forward measure that expresses uncertainty on a set B ⊆ B by the propagation of π through F , defined as F # π ( B ) = π ( F -1 ( B )) . Further, we assume all metrics on measures in this paper to be the total variation metric, and denote them by d both on Prob ( S ) and Prob (Ξ) . In general, we assume all maps to be Lipschitz continuous, and we omit Lipschitz constants in inequalities (details are provided in the Appendix).

Problem Formulation Consider the setting of a parametric PDE with forward solution mapping G : ξ ↦→ s , where ξ ∈ Ξ is a vector of finite-dimensional parameters and s ∈ S is a function-valued output. Since it is the goal to calibrate the input parameters, we omit any functional inputs to G , such

̸

as initial or boundary conditions, and assume they are kept constant or encoded in the parameters ξ . Given the forward operator G , the forward model uncertainty is quantified by propagating a given distribution of parameters ρ ∈ Prob(Ξ) through the model, i.e. evaluating the push-forward measure G # ρ . In practice, the parameters ξ are not available and need to be inferred from indirect measurements u ∈ U , which we assume to be functions in space or time (e.g., time series), and, in general, u = s . The extended solution operator ˜ G : u ↦→ s then maps between functional measurements and functional model output, omitting the intermediate parameter space. Since the measurements u are impacted by the uncertain parameters ξ , the function inputs u are as well equipped with uncertainty and represented by the measure µ ∈ Prob( U ) . Finally, the inverse problem consists in estimating the distribution ρ ( ξ | u ) of parameters given measurements u . The corresponding uncertainty on the predicted outcomes s is then given by the propagated uncertainty G # ρ ( . | u ) .

Unified Objective Given an approximate forward operator ˜ G θ ≈ ˜ G between function spaces, and an estimated distribution µ ϕ ≈ µ of input functions u , parameterized by θ and ϕ , respectively, we use triangle inequality to observe that

Thus, we found a unified objective consisting of two steps corresponding to the two terms in the right hand side (rhs) of Equation (1). In the measure matching step, the objective amounts to learn an approximation µ ϕ of the measure µ , whereas in the operator learning step, the objective is to learn a Neural Operator ˜ G θ that approximates the underlying ground truth operator ˜ G .

Our reformulation of operator learning naturally fits into the aim of this paper to propose a method that can act as an operator surrogate ( forward problem , operator learning objective) as well as performing parameter inference by minimizing distances on measures ( inverse problem , measure matching objective), unifying the apparently unrelated problems of surrogate modeling and inference.

Forward Problem: Operator Learning The operator ˜ G resembles the solution operator to a PDE that maps function inputs such as initial and boundary conditions, coefficients, sources etc to (observables of) the solution of the PDE. It is the goal of supervised operator learning [29] to learn this type of operator as a parameterized Neural Operator (NO) ˜ G θ on a (training) distribution µ ∈ Prob ( U ) by minimizing the operator learning objective in (1). This objective is bound by supervised learning on the parameter space Ξ ,

The loss is approximated by training samples of the form { ξ i , G ( ξ i ) } N i =1 , sampled from an underlying data distribution ρ ∈ Prob(Ξ) . In order to use a NO on finite-dimensional inputs, we define the band-limited lifting h θ 3 ( ξ ) , composed of a lifting increasing the dimension of the parameters and an inverse Fourier transform. We then choose to instantiate G θ based on Fourier Neural Operator (FNO) [31] layers K θ 2 , implemented with discrete spectral evaluations as in [34] to handle irregularly sampled measurements for the output function,

where Q θ 1 is a learnable map that projects the channels to the dimensions of the output function and θ = [ θ 1 , θ 2 , θ 3 ] are the trainable parameters.

Inverse Problem: Measure Matching We would like to approximate the true posterior measure ρ ( ξ | u ) given measurements u by ρ ϕ ( ξ | u ) . This task is equivalent to minimizing the measure matching objective in (1) since (Appendix A.7)

To minimize the latter distance, we adapt Flow-Matching Posterior Estimation (FMPE) to handle function-valued conditional inputs u . FMPE trains a flow function that maps samples from a standard

normal distribution to those from the target distribution ρ ( ξ | u ) . Unlike diffusion models [54, 20, 58], which require multiple steps, FMPE achieves this transformation in a single step, enabling faster training and evaluation. Both approaches are similar, but differ in their probability paths: diffusion models use a diffusion-based path, while FMPE employs an optimal transport path, leading to greater regularity and improved performance in low data settings.

The function input u is handed to the FMPE flow parameterized by v ϕ 0 t, ˆ u ( ξ ) as finite-dimensional conditional information ˆ u . We encode the infinite-dimensional u into ˆ u = T ϕ 1 ,ϕ 2 ( u ) as an adapted FNO, mimicking the forward surrogate. We project the functions to the parameters using a bandlimited forward Fourier transform ˜ h to create a fixed size latent representation of a function regardless of the input dimensions, subsequently projecting the large latent dimension to the smaller dimension m of ξ by FMPE,

where P ϕ 2 is a lifting layer and K ϕ 1 a concatenation of FNO layers. For the detailed formulation of the FMPE loss, we refer to the original reference [60] and equation (32) in Appendix A.8, and formulate it simply as

where ϕ = [ ϕ 0 , ϕ 1 , ϕ 2 ] are the joint trainable parameters of the flow itself and the FNO-based conditional information. The flow is then trained on data of the form { ( u i , ξ i ) } n i =1 .

Unified Training and Evaluation The proposed methodology FUSE obtains the optimally trained parameters θ and ϕ by minimizing the respective objectives of Equations (3) and (7),

thus minimizing the joint objective (1). For all experiments presented here, the two FUSE objectives are decoupled during training, implying that for each training batch, L 1 and L 2 are calculated and backpropagation is performed separately over each set of network parameters. This strategy aids the training, allowing us to avoid hyperparameter tuning of the loss function. It is possible to backpropagate losses through the entire network as discussed in Appendix A.1; however, this does not result in improvements at evaluation time, further supporting our claim that decoupled objectives may be unified at evaluation time to draw connections between forward and inverse problems.

When employing the trained FUSE model, we assume that measurements u are available and it is the goal to infer both the distribution of parameters ξ that best fit the data and the distribution of function outputs s that would result from those parameters. We obtain M parameter samples from the trained FMPE model as ξ i ∼ ρ ϕ ( ξ | u ) , i = 1 , ..., M , and pass these to the FNO forward surrogate to obtain M samples of the simulated outputs s i = G θ ( ξ i ) , i = 1 , ..., M , where we assume M sufficiently large to adequately sample the parameter distribution. We hence obtain an ensemble of estimates with mean prediction ¯ s = 1 M ∑ M i =1 s i and standard deviation σ s = ( 1 M -1 ∑ M i =1 ( s i -¯ s ) 2 ) 1 / 2 . While both model parts can also be evaluated separately, it is this propagated uncertainty on s that offers most insight in the effect of parametric uncertainty within the emulated models.

3 Experiments

For evaluation, we are given testing data of the form ( u, ξ ∗ , s ) obtained by numerical simulation, where we will refer to ξ ∗ as the data-generating parameters . Based on measurements u , the inverse FMPE model provides samples of the distribution of discrete parameters ξ i , that are compared to the true data-generating parameters ξ ∗ using the Continuous Ranked Probability Score (CRPS). The performance of the deterministic forward prediction is measured with the relative L 1 and L 2 errors over the continuous data and predictions. We compare FUSE against cVANO [26], inVAErt [57], and GAROM[10], and we consider an ablation where we substitute the FNO layers with UNet layers [47] in the FUSE model and leave out the Fourier transforms. In addition, we provide an ablation using a denoising diffusion probabilistic model (DDPM) [20] instead of FMPE in Table 5, and examine FMPE's performance with varying data sizes in Table 6. For details on the training and evaluation metrics, please refer to A.2.

PDE Test Cases As test cases, we consider a simulation of the pulse wave propagation (PWP) in the human arterial network and one of an atmospheric cold bubble (ACB). Both experiments are

described by a set of PDEs whose solutions are fully parameterized by finite-dimensional parameters encoding model properties, as well as boundary and initial conditions. In both of these experiments, to replicate a patient or downburst observed in the real world, the solver needs to be calibrated for precise conditions, e.g. a specific patient or atmospheric conditions, and forward uncertainty quantification of the calibrated predictions are of interest. Details on both cases are provided in Appendices A.3 and A.4.

Atmospheric Cold Bubble (ACB) In atmospheric modeling, the two-dimensional dry cold bubble [56] is a well-known test case for numerical simulations resembling downbursts, which cause extreme surface winds in thunderstorms. Within a domain of neutral atmospheric stability without background wind, an elliptic cold air anomaly is prescribed and initiates a turbulent flow as it sinks to the ground, see Figure A.12 for an example. Time series of horizontal and vertical velocity, u and w , are recorded at eight sensors placed inside the domain. Similar data is obtained by weather stations ( z ≈ 2 m) or turbulence flux towers ( z ≈ 10 -50 m), experienced by wind turbines or high-rise buildings ( z ≈ 100 m), or with unmanned aerial vehicles. The PDE model PyCLES [42] used for simulation is parameterized by the turbulent eddy viscosity ν t and diffusivity D t as model parameters, as well as four parameters describing the amplitude and shape of the initial cold perturbation ( a, x r , z r , z c ). The target task for FUSE here is to calibrate these parameters ξ ∈ R 6 to time series measurements u : [0 , T ] → R 20 , at ten locations for horizontal and vertical velocity each, where the input measurements used for calibration and the model output used for prediction and uncertainty quantification share the same space U = S of time series.

Pulse Wave Propagation (PWP) in the Human Body The pulse wave propagation in the cardiovascular system contains a great deal of information regarding the health of an individual. For this reason, there are efforts to measure and leverage PWP in both wearable devices, e.g. smart-watches, and clinical medicine. Different systemic parameters ξ ∈ R 32 such as stroke volume (SV), heart rate (HR), and patient age have been shown to affect the morphology of pulse waves [7]. These are used to parameterize pulse wave time series of pressure, velocity, and photoplethysmography (PPG) at 13 locations of systemic arteries of the human cardiovascular system through a reduced order PDE model in the dataset published by [7]. See Tables 7 and 8 in Section A.3 for the full set of parameters and locations. In clinical applications, only measurements u at easily accessible locations (such as the wrist) are available, while it is the goal to predict the pulse signal s at other locations of interest from the inferred parameters ξ . The space of input functions U hence differs from the space of output functions S in this test case. In order to account for different clinical scenarios, we train the FUSE model using random masking of locations and evaluate it on different levels of available input information,

Level 1. Perfect information:

Pressure, velocity, and PPG at all locations,

Level 2. Intensive care unit information: Pressure, velocity, and PPG at the wrist,

Level 3. Minimal information: PPG at the fingertip.

4 Results

Collected results for all experiments are summarized in Table 1. While the main tasks of the FUSE model are evaluated for both test cases, we focus on ACB to exemplify its generalization properties and on PWP for different levels of input information. We would like to emphasize that the goal of uncertainty representation here is to capture parametric uncertainty inherent to the data-generating numerical model and its parameterizations. Our model does not provide a notion of uncertainty in the predictions due to imperfect training of the neural networks. Given that these errors are generally low (Table 1), we are confident to interpret all spread on ξ and s given by FUSE as parametric uncertainty in the data-generating model. In particular, we expect the true posterior distributions to have positive spread, hence a prediction with small ensemble spread is not necessarily accurate.

Inverse Problem Given time series measurements u , the inverse model samples from the distribution of parameters, ξ ∼ ρ ϕ ( ξ | u ) , with more samples in areas of the parameter space that are more likely to match the data-generating parameters ξ ∗ . We find that FUSE outperforms the other methods when sufficient input information is given. We observe that inVAErt experiences posterior collapse for PWP in this setting, which is represented in high CRPS (Figure A.9). Histograms reveal that FUSE captures the expected dependencies of the data on the parameters (Figures A.1, A.2), such as wider distributions for scarcer information in PWP. In the ACB case, sharp Dirac-like estimates are obtained

for most of the samples (Figure A.14). However, a good fit for the numerical model parameters seems to rely on an accurate estimate of the initial condition (Figure A.16).

Forward Problem The surrogate G θ ( ξ ) is tasked to assess the ability of FUSE to predict the continuous s from the data-generating parameters ξ ∗ . FUSE consistantly outperforms all other methods in both test cases. In particular, FUSE is able to reproduce very nonlinear dynamics and sharp velocity gradients in the ACB case (Figure A.15). Potential errors mostly lie in the bulk velocity of the anomaly, resulting in a stretching or squeezing of the time series at later times. As expected, predictions are slightly worse further away from the perturbation location, where the flow has become more turbulent, especially for the vertical velocity which exhibits strong oscillations (Figure A.21a). For the PWP, continuous predictions based on the true parameters have an L1 error of well below 0 . 3% at all locations (Figures A.6, A.7). Smaller arteries generally exhibit a slightly higher error variance, which fits the physical problem interpretation, as smaller diameters show more variations to pressure than larger system arteries.

Unified Uncertainty Propagation Given an input u , the concatenated evaluation of the inverse and forward parts of FUSE provides a distribution over the output functions s , by first sampling parameters ξ i and then evaluating the forward emulator on this ensemble, yielding continuous samples s i . This can be interpreted as the uncertainty introduced by simulating the data with the parameterized model FUSE was trained on: Uncertainty is introduced as the parameters are not fully identifiable in the inverse step, and is propagated and expanded onto the continuous output through the ensemble predictions on the parameter samples. Again, FUSE outperforms all other models significantly in all experiments in the L 1 and L 2 error over the mean predictions ¯ s . Figures 2 and 3 show how FUSE captures both smooth and rough structures, even on the sample it performs worst on. It only exhibits slight difficulties in predicting a constant velocity close to zero for ACB. At each location of the ACB, the L 1 error obtained with the propagated ensemble mean ¯ s is comparable to the error by evaluating only the forward model, s = G θ ( ξ ∗ ) (Figure A.21b). An inspection of the predictions for all locations for the PWP case shows that the true time series lie within one standard deviation of the FUSE ensemble mean for the median sample, and within the min-max uncertainty bounds for the worst sample (Figures A.3, A.4). We observe that the accurate prediction of model parameters correlates to accurate predictions of pressures (Figure A.11).

0

200

400

600

800

Time (s)

-

4

-

2

0

2

4

w [m/s]

True

FUSE: Mean

Uncertainty Range

(a) FUSE

0

200

400

600

800

Time (s)

-

2

-

1

0

1

2

3

4

5

w [m/s]

True

UNet: Mean

Uncertainty Range

(b) UNet

0

200

400

600

800

Time (s)

-

2

-

1

0

1

2

3

4

5

w [m/s]

True

InVAErt: Mean

Uncertainty Range

(c) InVAErt

0

200

400

600

800

Time (s)

-

2

-

1

0

1

2

3

4

5

w [m/s]

True

cVANO: Mean

Uncertainty Range

(d) cVANO

0

200

400

600

800

Time (s)

-

2

-

1

0

1

2

3

4

5

w [m/s]

True

GAROM: Mean

Uncertainty Range

(e) GAROM*

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

80

90

100

110

120

130

140

True Value

FUSE Mean

Min/Max

(a) FUSE

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

60

70

80

90

100

110

120

130

140

True Value

UNet Mean

Min/Max

(b) UNet

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

70

80

90

100

110

120

130

True Value

InVAErt Mean

Min/Max

(c) InVAErt

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

60

70

80

90

100

110

120

130

140

True Value

cVANO Mean

Min/Max

(d) cVANO

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

90

100

110

120

130

True Value

GAROM Mean

Min/Max

(e) GAROM*

Sensitivity Analysis Based on the good approximative properties showcased above, the forward operator G θ of FUSE can be used as a fast surrogate model to assess the sensitivity of the underlying PDE to varying the parameters ξ . Such analysis is usually constrained to a small sample size due to the large computational costs, and the dense sampling enabled by the surrogate helps to explore the parameter space for parameters that optimally fit patient data in PWP, or exhibit extreme winds in ACB. Varying one parameter at a time, while keeping all others fixed at their default value, exhibits the single effect on the data ( fingerprint ). For ACB, an increased amplitude of the cold anomaly mainly results in a speed-up and hence a squeezing of the time series, while other parameters have more nonlinear effects (Figure A.17). As the FUSE model provides no access to the full velocity fields, only numerical simulations reveal that these sharp sensitivities are mainly caused by certain features (eddies) of the flow reaching or missing a sensor depending on the parameter value. A validation of this sensitivity analysis for an estimate of the maximal horizontal velocity over the time series is provided in Figure 4 for a pairwise fingerprint with 10 , 000 FUSE samples, with consistently low errors compared to 100 numerical samples. Only for one sample with a weak and small perturbation, the signal is not captured accurately. An additional evaluation of 60 samples at the margins of the parameter space shows that the peak velocities are consistently represented well (Figures A.20, A.19). For PWP, the sensitivity results (Figure A.10) are consistent with observations reported in the literature [7].

30.0

60

30.0

0.20

relative error

25.0

50

25.0

0.15

aplitude

20.0

40

amplitude

20.0

lue

15.0

30

15.0

0.10

:[s/u]

10.0

20

10.0

0.05

zoun

10

5.0

5.0

0.00

25.

diffusivity

50.

6

diffusivity

Out of Distribution Generalization for ACB We test the generalization capabilities of the FUSE forward model on test samples with amplitudes larger than seen in training in the pairwise sensitivity setup (Figure 4). The FUSE model is able to capture the shape of the pairwise parameter dependencies but struggles to locate the sharp dependence of the local peak velocity on the eddy diffusivity. Averaged errors in the out of distribution range are shown in Table 2, with FUSE performing clearly superior to all other models.

Levels of Available Information for PWP The case of missing input information is modeled by masking certain input components when evaluating the models. While FUSE shows regular performance comparable with the baselines on levels 2 and 3 of missing information, cVANO seems to be better suited for this setting (Table 1), probably due to the assumption that the latent dimensions are uncorrelated. As a sanity check, FUSE samples the prior distribution of the training data when evaluated without any conditional input information u (Figure A.8). When the available input measurement locations of the PWP experiment are known a priori, it is possible to get more accurate results when training FUSE without masking. For example, training to predict pressures and parameters from only the PPG data at the fingertip only, without masking, results in a CRPS of 4 . 28 × 10 -2 and a relative L 1 error of 3.6%, roughly half the error of the masked model. If the measurement data is fixed, predictions without masking are much more suitable for analysis.

5 Discussion

Summary We propose FUSE, a framework unifying surrogate modeling and parameter identification for parametric PDEs by bridging operator learning with flow-matching posterior estimation. Both are represented in the FUSE objective through a deterministic forward loss and a probabilistic inverse loss, respectively. The joint architecture allows for inverse estimation of scalar parameters ξ given

continuous measurements u , as well as forward predictions of (other) continuous measurements s given ξ . Inheriting from the properties of its components, FUSE is discretization invariant with respect to the continuous inputs and outputs, and can represent arbitrary distributions in the fully interpretable parameter space. As this latent space is defined through a known parameterization of the underlying PDE, FUSE allows for accurate calibration and evaluation of a numerical solver, as well as performing downstream tasks such as parameter uncertainty quantification. On the parametric PDEs of pulse wave propagation (PWP) in the human cardiovascular system and an atmospheric cold bubble (ACB) with time series measurements, FUSE consistently outperforms four baseline methods designed to perform similar tasks. It also shows good out-of-distribution generalization and great flexibility when trained and evaluated on different levels of available input information.

Connection to Existing Methods The proposed methodology and the adapted implementation of the FNO and FMPE offer the following advantages over the existing methods for joint parameter estimation and forward emulation mentioned in the introduction. By incorporating Neural Operators in both the forward and inverse parts, we ensure discretization invariance throughout the model [3]. In contrast to Variational Autoencoders, we present a fully probabilistic formulation and require no restriction on the latent data distribution. This allows us to model arbitrary posterior distributions, whereas cVANO is restricted to Gaussian distributions. Likewise, FUSE can leverage powerful, existing neural operators which improves predictions on the output functions. The experiments have shown that FUSE excels over all baselines in both test cases. In particular, cV ANO and GAROM fail to capture even rough characteristics of the flow in the forward emulation for the ACB case with nonlinear dynamics, and InVAErt shows a consistent bias towards the mean in the PWP task. Only in the case of missing information in the smoother PWP case, cVANO outperforms FUSE on the inverse problem. In terms of the choice of FNO in both the forward and inverse part of FUSE, the ablation with UNet layers exhibits larger uncertainties and worse performance of the predicted mean. An additional ablation, presented in the Appendix 5, shows that optimal transport probability paths perform better than the diffusion-based paths in a conditional DDPM model. Finally, Gaussian processes are often chosen for their ability to handle unstructured data. We incorporate this property by using the grid-independent FNO implementation by [34].

Applicability FUSE is explicitly formulated for finite-dimensional parameters and function measurements and outputs. This setting naturally arises when calibrating a numerical solver, with training data generated synthetically by the solver itself, and was demonstrated in two test cases. However, it is very common for real measured datasets, e.g. in bioengineering [22], and more specifically when involving PWP, to contain both time series data and vectors of parameters available for different patients. FUSE naturally extends to these datasets, where an underlying PDE is assumed but not formulated or simulated explicitly, including parameter inference using real data [59], precision medicine or solver calibration [46], and fingerprinting to discover parameter-disease correlations [55]. In the case that the parameters are non-physical, and hence not measurable, such as the numerical model parameters in the ACB case, this does of course not apply, and the main aim of solving the forward and inverse problem is to explore the solver itself. Moreover, FUSE may help to refine existing parameterizations either through disentanglement [18] or by identifying parameters with little influence on the simulated data.

Limitations and Future Work The FUSE framework aims to emulate a given parameterization of a PDE. If such a parameterization (or a corresponding dataset) is not available, VAE-based preprocessing [4] or manifold discovery [5] may be used. When applying FUSE to real measurements, the parameterization turns to be only an approximation of the dynamical system, whereas it is considered a perfect model in the experiments presented here. This structural model uncertainty has to be added to the interpretation of the unified uncertainties given by the FUSE model, as it has to be for any other inverse algorithm. In practice, measurements are always associated with a measurement error, which is considerable in particular for extreme measurements. The current implementation of FUSE does not allow for errors in the continuous inputs to be taken into account, and it is left for a future extension of the framework to include these into a fully Bayesian formulation of the inverse model. In spatially distributed applications, such as atmospheric modeling, the parameters of interest might be themselves space/time-dependent functions, such as surface properties or initial conditions [48]. Based on the scaling properties of the FMPE model for high-dimensional data such as images [35], FUSE is expected to scale well in size for high-dimensional parameter spaces, but is yet to be extended to a resolution-invariant function representation in latent space. These limitations will be addressed in future research.

Acknowledgments

G.K. would like to acknowledge the support from Asuera Stiftung via the ETH Zurich Foundation.

References

J. Alastruey, K. H. Parker, S. J. Sherwin, et al. Arterial pulse wave haemodynamics. In 11th international conference on pressure surges , volume 30, pages 401-443. Virtual PiE Led t/a BHR Group Lisbon, Portugal, 2012.

A. Arzani, J.-X. Wang, M. S. Sacks, and S. C. Shadden. Machine learning for cardiovascular biomechanics modeling: challenges and beyond. Annals of Biomedical Engineering , 50(6): 615-627, 2022. doi:10.1007/s10439-022-02967-4.

F. Bartolucci, E. de Bezenac, B. Raonic, R. Molinaro, S. Mishra, and R. Alaifari. Representation equivalent neural operators: a framework for alias-free operator learning. Advances in Neural Information Processing Systems , 36, 2024. URL https://papers.nips.cc/paper_files/ paper/2023/hash/dc35c593e61f6df62db541b976d09dcf-Abstract-Conference. html .

P. Bojanowski, A. Joulin, D. Lopez-Pas, and A. Szlam. Optimizing the latent space of generative networks. In Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pages 600-609. PMLR, 10-15 Jul 2018. URL https://proceedings.mlr.press/v80/bojanowski18a.html .

J. Brehmer and K. Cranmer. Flows for simultaneous manifold learning and density estimation. In Advances in Neural Information Processing Systems (NeurIPS) , 2020. URL https://proceedings.neurips.cc/paper/2020/hash/ 051928341be67dcba03f0e04104d9047-Abstract.html .

S. ˇ Cani´ c, J. Tambaˇ ca, G. Guidoboni, A. Mikeli´ c, C. J. Hartley, and D. Rosenstrauch. Modeling viscoelastic behavior of arterial walls and their interaction with pulsatile blood flow. SIAM Journal on Applied Mathematics , 67(1):164-193, 2006. doi:10.1137/060651562.

P. H. Charlton, J. Mariscal Harana, S. Vennin, Y . Li, P. Chowienczyk, and J. Alastruey. Modeling arterial pulse waves in healthy aging: a database for in silico evaluation of hemodynamics and pulse wave indexes. American Journal of Physiology-Heart and Circulatory Physiology , 317 (5):H1062-H1085, 2019. doi:10.1152/ajpheart.00218.2019.

P. Y. Chen, J. Xiang, D. H. Cho, Y. Chang, G. A. Pershing, H. T. Maia, M. M. Chiaramonte, K. T. Carlberg, and E. Grinspun. CROM: Continuous reduced-order modeling of PDEs using implicit neural representations. In The Eleventh International Conference on Learning Representations , 2023. URL https://iclr.cc/virtual/2023/poster/12094 .

E. Cleary, A. Garbuno-Inigo, S. Lan, T. Schneider, and A. M. Stuart. Calibrate, emulate, sample. Journal of Computational Physics , 424:109716, 2021. ISSN 00219991. doi:10.1016/j.jcp.2020.109716. URL https://www.sciencedirect.com/science/ article/pii/S0021999120304903 .

D. Coscia, N. Demo, and G. Rozza. Generative adversarial reduced order modelling. Sci Rep , 14(3826), 2024. doi:10.1038/s41598-024-54067-z.

L. Dinh, D. Krueger, and Y. Bengio. Nice: Non-linear independent components estimation. In International Conference on Learning Representations , 2015. URL https://arxiv.org/ abs/1410.8516 .

L. Dinh, J. Sohl-Dickstein, and S. Bengio. Density estimation using real NVP. In The Fifth International Conference on Learning Representations , 2017. URL https://openreview. net/forum?id=HkpbnH9lx .

L. Formaggia, A. Quarteroni, and A. Veneziani. Cardiovascular Mathematics: Modeling and simulation of the circulatory system , volume 1. Springer Science & Business Media, 2010.

M. Gloeckler, M. Deistler, C. D. Weilbach, F. Wood, and J. H. Macke. All-in-one simulationbased inference. In Proceedings of the 41st International Conference on Machine Learning , volume 235, pages 15735-15766. PMLR, 21-27 Jul 2024. URL https://proceedings.mlr. press/v235/gloeckler24a.html .

M. Gulian, A. Frankel, and L. Swiler. Gaussian process regression constrained by boundary value problems. Computer Methods in Applied Mechanics and Engineering , 388:114117, 2022. doi:https://doi.org/10.1016/j.cma.2021.114117.

R. Halder, M. Damodaran, and B. C. Khoo. Deep learning based reduced order model for airfoil-gust and aeroelastic interaction. AIAA Journal , 58(4):1595-1606, 8 2020. doi:10.2514/1.J059027.

Z. Hao, Z. Wang, H. Su, C. Ying, Y. Dong, S. Liu, Z. Cheng, J. Song, and J. Zhu. GNOT: A general neural operator transformer for operator learning. In Proceedings of the 40th International Conference on Machine Learning , volume 202, pages 12556-12569. PMLR, 23-29 Jul 2023. URL https://proceedings.mlr.press/v202/hao23c.html .

I. Higgins, L. Matthey, A. Pal, C. P. Burgess, X. Glorot, M. M. Botvinick, S. Mohamed, and A. Lerchner. beta-VAE: Learning basic visual concepts with a constrained variational framework. In The Fifth International Conference on Learning Representations , volume 3, 2017. URL https://openreview.net/forum?id=Sy2fzU9gl .

S. Hijazi, M. Freitag, and N. Landwehr. POD-galerkin reduced order models and physicsinformed neural networks for solving inverse problems for the navier-stokes equations. Adv. Model. and Simul. in Eng. Sci. , 10(5):5, 03 2023. doi:10.1186/s40323-023-00242-2.

J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems , volume 33, pages 6840-6851. Curran Associates, Inc., 2020. URL https://papers.nips.cc/paper_files/paper/2020/hash/ 4c5bcfec8584af0d967f1ab10179ca4b-Abstract.html .

P. Jin, S. Meng, and L. Lu. MIONet: Learning multiple-input operators via tensor product. SIAM Journal on Scientific Computing , 44(6):A3490-A3514, 2022. doi:10.1137/22M1477751.

A. E. W. Johnson, T. J. Pollard, L. Shen, H. L. Li-wei, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. A. Celi, and R. G. Mark. MIMIC-III, a freely accessible critical care database. Scientific data , 3:160035, 2016. doi:10.1038/sdata.2016.35.

B. Kim, V. C. Azevedo, N. Thuerey, T. Kim, M. Gross, and B. Solenthaler. Deep fluids: A generative network for parameterized fluid simulations. Computer graphics forum , 38(2):59-70, 2019. doi:10.1111/cgf.13619.

D. P. Kingma and P. Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018. URL https://papers.nips.cc/paper_files/paper/2018/hash/ d139db6a236200b21cc7f752979132d0-Abstract.html .

D. P. Kingma and M. Welling. Auto-encoding variational bayes. In International Conference on Learning Representations , 2013. URL https://arxiv.org/abs/1312.6114 .

G. Kissas. Towards Digital Twins for Cardiovascular Flows: A Hybrid Machine Learning and Computational Fluid Dynamics Approach . PhD thesis, University of Pennsylvania, 2023.

G. Kissas, J. H. Seidman, L. F. Guilhoto, V. M. Preciado, G. J. Pappas, and P. Perdikaris. Learning operators with coupled attention. J. Mach. Learn. Res. , 23(1), Jan. 2022. URL https://dl.acm.org/doi/10.5555/3586589.3586804 .

N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart, and A. Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research , 24(89):1-97, 2023. URL http://jmlr.org/papers/v24/ 21-1524.html .

S. Lanthaler, S. Mishra, and G. E. Karniadakis. Error estimates for DeepONets: A deep learning framework in infinite dimensions. Transactions of Mathematics and Its Applications , 6(1): tnac001, 2022. doi:10.1093/imatrm/tnac001.

M. Letafati, S. Ali, and M. Latva-aho. Conditional denoising diffusion probabilistic models for data reconstruction enhancement in wireless communications, 2024. URL https://arxiv. org/abs/2310.19460 .

Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations , volume 2010.08895, 2021. URL https://iclr.cc/virtual/ 2021/poster/3281 .

Z. Li, K. Meidani, and A. B. Farimani. Transformer for partial differential equations' operator learning. Transactions on Machine Learning Research , 2023. ISSN 2835-8856. URL https: //openreview.net/forum?id=EPPqt3uERT .

Z. Li, D. Shu, and A. Barati Farimani. Scalable transformer for pde surrogate modeling. In Advances in Neural Information Processing Systems , volume 36, pages 28010-28039. Curran Associates, Inc., 2023. URL https://papers.nips.cc/paper_files/paper/2023/ hash/590daf74f99ee85df3d8c007df9c8187-Abstract-Conference.html .

L. E. Lingsch, M. Y. Michelis, E. De Bezenac, S. M. Perera, R. K. Katzschmann, and S. Mishra. Beyond regular grids: Fourier-based neural operators on arbitrary domains. In Proceedings of the 41st International Conference on Machine Learning , volume 235, pages 30610-30629. PMLR, 21-27 Jul 2024. URL https://proceedings.mlr.press/v235/lingsch24a.html .

Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling. In International Conference on Learning Representations , 2023. URL https: //iclr.cc/virtual/2023/poster/11309 .

S. Liu, C. Zeman, S. L. Sørland, and C. Schär. Systematic Calibration of a Convection-Resolving Model: Application Over Tropical Atlantic. Journal of Geophysical Research: Atmospheres , 127(23), 2022. doi:10.1029/2022JD037303.

L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence , 3(3):218-229, Mar. 2021. doi:10.1038/s42256-021-00302-5.

A. Moore. Investigating the Near-Surface Wind Fields of Downbursts using a Series of HighResolution Idealized Simulations. Weather and Forecasting , 2024. doi:10.1175/WAF-D-230164.1.

L. Orf, E. Kantor, and E. Savory. Simulation of a downburst-producing thunderstorm using a very high-resolution three-dimensional cloud model. Journal of Wind Engineering and Industrial Aerodynamics , 2012. doi:10.1016/j.jweia.2012.02.020.

A. Parodi, M. Lagasio, M. Maugeri, B. Turato, and W. Gallus. Observational and Modelling Study of a Major Downburst Event in Liguria: The 14 October 2016 Case. Atmosphere , 10(12): 788, 2019. doi:10.3390/atmos10120788.

O. Pauluis. Thermodynamic Consistency of the Anelastic Approximation for a Moist Atmosphere. Journal of the Atmospheric Sciences , 65(8), 2008. doi:10.1175/2007JAS2475.1.

K. G. Pressel, C. M. Kaul, T. Schneider, Z. Tan, and S. Mishra. Large-eddy simulation in an anelastic framework with closed water and entropy balances. Journal of Advances in Modeling Earth Systems , 7(3):1425-1456, 2015. ISSN 1942-2466. doi:10.1002/2015MS000496. URL https://onlinelibrary.wiley.com/doi/abs/10.1002/2015MS000496 .

A. Quarteroni, A. Manzoni, and F. Negri. Reduced basis methods for partial differential equations: an introduction , volume 92. Springer, 2015.

B. Raonic, R. Molinaro, T. De Ryck, T. Rohner, F. Bartolucci, R. Alaifari, S. Mishra, and E. de Bézenac. Convolutional neural operators for robust and accurate learning of pdes. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems , volume 36. Curran Associates, Inc., 2023. URL https://papers.nips.cc/paper_files/paper/2023/hash/ f3c1951b34f7f55ffaecada7fde6bd5a-Abstract-Conference.html .

D. Rezende and S. Mohamed. Variational inference with normalizing flows. In F. Bach and D. Blei, editors, Proceedings of the 32nd International Conference on Machine Learning , volume 37, Lille, France, 07-09 Jul 2015. PMLR. URL https://proceedings.mlr.press/ v37/rezende15.html .

J. Richter, J. Nitzler, L. Pegolotti, K. Menon, J. Biehler, W. A. Wall, D. E. Schiavazzi, A. L. Marsden, and M. R. Pfaller. Bayesian windkessel calibration using optimized 0d surrogate models, 2024. URL https://arxiv.org/abs/2404.14187 .

O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 , pages 234-241, Cham, 2015. Springer International Publishing. doi:10.1007/978-3-31924574-4_28.

Y. Ruckstuhl and T. Janji´ c. Combined State-Parameter Estimation with the LETKF for Convective-Scale Weather Forecasting. Monthly Weather Review , 148(4):1607-1628, 2020. doi:10.1175/MWR-D-19-0233.1.

P. Segers, E. Rietzschel, M. De Buyzere, N. Stergiopulos, N. Westerhof, L. Van Bortel, T. Gillebert, and P. Verdonck. Three-and four-element windkessel models: assessment of their fitting performance in a large cohort of healthy middle-aged individuals. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine , 222(4):417-428, 2008. doi:10.1243/09544119JEIM287.

J. Seidman, G. Kissas, P. Perdikaris, and G. J. Pappas. NOMAD: Nonlinear manifold decoders for operator learning. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35. Curran Associates, Inc., 2022. URL https://papers.nips.cc/paper_files/paper/2022/hash/ 24f49b2ad9fbe65eefbfd99d6f6c3fd2-Abstract-Conference.html .

J. H. Seidman, G. Kissas, G. J. Pappas, and P. Perdikaris. Variational autoencoding neural operators. In Proceedings of the 40th International Conference on Machine Learning , volume 202. PMLR, 23-29 Jul 2023. URL https://proceedings.mlr.press/v202/seidman23a. html .

S. Sherwin, L. Formaggia, J. Peiro, and V. Franke. Computational modelling of 1D blood flow with variable mechanical properties and its application to the simulation of wave propagation in the human arterial system. International Journal for Numerical Methods in Fluids , 43(6-7): 673-700, 2003. doi:10.1002/fld.543.

Y. Shi, A. F. Gao, Z. E. Ross, and K. Azizzadenesheli. Universal functional regression with neural operator flows. In Advances in Neural Information Processing Systems:Workshop on Bayesian Decision-making and Uncertainty , 2024. URL https://openreview.net/forum? id=ZCtnWuaZiI .

J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning , volume 37 of Proceedings of Machine Learning Research , Lille, France, 0709 Jul 2015. PMLR. URL https://proceedings.mlr.press/v37/sohl-dickstein15. html .

R. Sokolow, G. Kissas, C. Beeche, S. Swago, E. W. Thompson, M. Viswanadha, J. Chirinos, S. Damrauer, P. Perdakaris, D. J. Rader, and W. R. Witschey. An aortic hemodynamic fingerprint reduced order modeling analysis reveals traits associated with vascular disease in a medical biobank. bioRxiv , 2024. doi:10.1101/2024.04.19.590260.

J. M. Straka, R. B. Wilhelmson, L. J. Wicker, J. R. Anderson, and K. K. Droegemeier. Numerical solutions of a non-linear density current: A benchmark solution and comparisons. International Journal for Numerical Methods in Fluids , 17(1):1-22, 1993. doi:10.1002/fld.1650170103.

G. G. Tong, C. A. Sing Long, and D. E. Schiavazzi. InV AErt networks: A data-driven framework for model synthesis and identifiability analysis. Computer Methods in Applied Mechanics and Engineering , 423:116846, 2024. doi:https://doi.org/10.1016/j.cma.2024.116846.

R. Wang, Z. Chen, Q. Luo, and F. Wang. A conditional denoising diffusion probabilistic model for radio interferometric image reconstruction. In Proceedings of the 11th International Conference on Artificial Intelligence and Applications (FAIA) , 2023. doi:10.3233/FAIA230554.

A. Wehenkel, J. Behrmann, A. C. Miller, G. Sapiro, O. Sener, M. C. Cameto, and J.-H. Jacobsen. Simulation-based inference for cardiovascular models. In NeurIPS Workshop , 2024. URL https://arxiv.org/abs/2307.13918 .

J. Wildberger, M. Dax, S. Buchholz, S. Green, J. H. Macke, and B. Schölkopf. Flow matching for scalable simulation-based inference. In Advances in Neural Information Processing Systems , volume 36. Curran Associates, Inc., 2023. URL https://papers.nips.cc/paper_files/ paper/2023/hash/3663ae53ec078860bb0b9c6606e092a0-Abstract-Conference. html .

J. Xiong, X. Cai, and J. Li. Clustered active-subspace based local gaussian process emulator for high-dimensional and complex computer models. Journal of Computational Physics , 450, 2022. doi:10.1016/j.jcp.2021.110840.

J. Zhang, S. Zhang, and G. Lin. PAGP: A physics-assisted gaussian process framework with active learning for forward and inverse problems of partial differential equations, 2022. URL http://arxiv.org/abs/2204.02583 .

A Supplementary Material

A.1 Training Details

All experiments were performed on an Nvidia GeForse RTX 3090 GPU with 24GB of memory. The number of training, validation, and test samples are provided in Table 3, along with the data dimensions. Further details regarding the experimental data are provided in Sections A.3 and A.4.

Training Strategy For each experiment, we performed a hyperparameter sweep when possible to select the learning rate, scheduler rate, weight decay, and network size. Due to the large number of hyperparameters, we selected 64 random combinations of hyperparameters and trained each model for 500 epochs. Following the sweep, we selected the model with the best performance and trained it for 2000 epochs, saving it at the epoch with the best performance on the validation set. Following this, we evaluated its performance on the test samples. A special exception must be made for the V AE-based models, InVAErt and cVANO. InVAErt was particularly susceptible to posterior collapse. Following the advice of the authors in Section 2.3.2 of [57], we hand-tuned the model to prevent over-parameterization and used early stopping. Similarly, cV ANO was susceptible to training instability and exploding gradients, which also necessitated careful tuning and training in order to get the best results. The final size of each model, in number of network parameters, is provided in Table 4. The results from the model selection revealed all models are fairly consistent in size, typically all on the same order of magnitude.

We observed some sensitivity to the normalization scheme used on the data. In the PWP experiment, we normalize each input channel to a range between zero and one by dividing by the maximum value across all samples at that channel. In the ACB experiment, we normalize each channel by taking the minimum and maximum value across all samples in that channel and rescaling these to a range between zero and one.

The training time per epoch of all experiments is consistent, ranging from 1.5 to 2 seconds. During the hyperparameter sweep, each model is trained for 500 epochs, taking approximately 12.5 minutes. The best model from the sweep is trained for a total of 2000 epochs, taking approximately 1 hour. Throughout this entire project, we estimate the total number of GPU-hours for all model hyperparameter sweeps, hand-tuning, and training of the final model to be approximately 180 GPU-hours.

Additional Training Approaches Although the two objectives of FUSE are decoupled during training time, it is also possible to backpropagate losses on the outputs through both the forward and inverse model components. FMPE supports differentiable sampling, therefore the loss on the outputs s could be formulated as a function of continuous inputs u ,

In the case where the inputs and outputs, respectively u and s , are identical, a reconstruction loss may be applied to further train the inverse model component. This entails predicting outputs s from the data-generating parameters ξ ∗ and predicting the inverse problem from the predictions s , as opposed to the original inputs u . This is formulated as

Although each approach has the potential to aid training it is still required to train each objective separately. Likewise, we did not observe improvements in our experiments by implementing these additional losses. This further supports our claim in Equation 1, that model components trained by decoupled objectives for forward and inverse problems may be unified at evaluation to propagate uncertainties in the parameters to outputs and provide interpretable results for complex and dynamic systems of PDEs.

A.2 Evaluation Metrics

Continuous Ranked Probability Score We define the Continuous Ranked Probability Score (CRPS) and present how it compares a single ground truth value y to a Cumulative Distribution Function (CDF) F , quantifying how well a distribution of samples fits a single target. The CRPS is

defined as

with independent and identically distributed random variables X and X ′ following the distribution dictated by F . The first expectation computes the mean absolute error of the samples compared to the target value, and the second expectation penalizes tight distributions which are far from the true value. A CRPS of zero indicates that the distribution perfectly fits the target sample by returning a Dirac measure.

L p Error We employ the relative L 1 and L 2 error for evaluating predictions of time series data. This metric is calculated by

where G ( ξ ) is the true function described by the system of PDEs, and G θ ( ξ ) is a prediction produced by the network.

A.3 Pulse Wave Propagation: Reduced Order Model for Blood Flow

The reduced order Navier-Stokes model considers a system of discrete compliant tubes and arterial segments, connected at points [1]. The length of the vessels is considered much larger than the local curvature and the wave propagation happens on the axial direction. Therefore, their properties can be described using a Cartesian coordinate x [52]. The luminal cross-sectional area is defined as A ( x, t ) = ∫ A dσ , the average velocity is defined as U ( x, t ) = 1 A ∫ S u ( x , t ) · n dσ and the volume flux at a given cross section as Q ( x, t ) = UA [1]. The arteries are allowed to deform in the radial direction due to the internal pressure P ( x, t ) which is considered constant over a cross section and the wall is considered impermeable. The blood is considered as an incompressible Newtonian fluid, with viscosity µ = 2 . 5 mPa s and density ρ = 1060 kg m -3 [13]. All the gravitational effects are ignored assuming a patient in a supine position [1]. Using the above assumptions, the pulse wave propagation is formulated as the following system of hyperbolic constitutive laws [1, 13]:

where a ( x, t ) = 1 AU 2 ∫ A u 2 dσ the Coriolis coefficient that accounts for the non-linearity of the integration over the cross section and describes the shape of the velocity profile. The velocity profile is commonly considered as axisymmetric, constant, and satisfying the non-slip boundary conditions on the arterial walls. An example of such profile is [1]:

where r is the radial coordinate, R ( x, t ) the lumen radius and ζ = 2 -α α -1 a constant. A flat profile is defined by α = 1 [7]. Integrating the three-dimensional Navier-Stokes equations for the axisymmetric flow provides the friction force per unit length f ( x, t ) = -2( ζ +2) µπU . To account for the fluid-structure interaction of the problem, a pressure area relation is derived by considering the tube law. For a thin isotropic homogenous and incompressible vessel that deforms axisymmetrically at each circular cross section the following Voigt-type visco-elastic law [6] is typically used:

where P e is the elastic pressure component, h ( x ) the wall thickness, E ( x ) the Young's modulus, ϕ ( x ) the wall viscosity, β ( x ) a wall stiffness coefficient, and Γ( x ) the wall viscosity. The scalars P ext and A 0 correspond to the external pressure and equilibrium cross-sectional area, respectively. The parameters β ( x ) and Γ( x ) are computed using empirical relations found in the literature [7]. For this problem, a pulse wave from the heart is considered as a boundary condition for the inlet of the artery. For merging and splitting of arteries, the conservation of mass and the continuity of dynamic and kinematic pressure are considered, as well as no energy losses. For the outlet of the arterial system, boundary conditions that model the downstream circulation are considered. In [7], the three-element Windkessel model [49] is used:

In [7], the authors consider algebraic relations from the literature that allow them to include different parameters, e.g. the volume flux, in the pulse wave propagation problem. For example, the digital PPG measurements can be related to the volume flux at the extremities as:

where T is the duration of a cardiac cycle, Q W the flow entering the Windkessel model, and Q out the flow in the outlet.

(c) Information level 3

(b) Information level 2

(a) Information level 1

20

40

60

80

0

25

age

60

80

0

20

HR

40

60

80

0

20

SV

4

6

0

20

CO

250

300

0

20

LVET

500

1000

0

20

dpdt

78

79

80

0

250

PFT

0

.

7

0

.

8

0

.

9

0

200

RFV

100

150

0

20

SBP

a

60

80

0

20

DBP

a

80

100

0

20

MAP

a

50

100

0

20

PP

a

100

150

0

20

SBP

b

60

80

0

20

DBP

b

80

100

0

20

MBP

b

50

100

0

25

PP

b

1

.

0

1

.

5

2

.

0

0

25

PP

amp

0

50

0

25

AP

0

50

0

20

AIx

50

100

150

0

25

Tr

5

10

15

0

20

PWV

a

5

10

15

0

20

PWV

cf

10

15

0

25

PWV

br

5

10

15

0

20

PWV

fa

35

40

45

0

50

dia

asca

25

30

0

25

dia

dta

12

.

5

15

.

0

17

.

5

0

25

dia

abda

8

10

0

25

dia

car

80

100

0

25

2

4

0

20

drop

fin

5

10

0

25

drop

ankle

100

200

300

0

25

SVR

20

40

60

80

0

20

age

60

80

0

25

HR

40

60

80

0

20

SV

4

6

0

20

CO

250

300

0

20

LVET

500

1000

0

20

dpdt

78

79

80

0

50

PFT

0

.

7

0

.

8

0

.

9

0

50

RFV

100

150

0

20

SBP

a

60

80

0

20

DBP

a

80

100

0

20

MAP

a

50

100

0

20

PP

a

100

150

0

20

SBP

b

60

80

0

20

DBP

b

80

100

0

20

MBP

b

50

100

0

20

PP

b

1

.

0

1

.

5

2

.

0

0

20

PP

amp

0

50

0

20

AP

0

50

0

25

AIx

50

100

150

0

25

Tr

5

10

15

0

25

PWV

a

5

10

15

0

20

PWV

cf

10

15

0

20

PWV

br

5

10

15

0

20

PWV

fa

35

40

45

0

20

dia

asca

25

30

0

20

dia

dta

12

.

5

15

.

0

17

.

5

0

20

dia

abda

8

10

0

20

dia

car

80

100

0

20

2

4

0

20

drop

fin

5

10

0

20

drop

ankle

100

200

300

0

20

SVR

20

40

60

80

0

20

age

60

80

0

25

HR

40

60

80

0

20

SV

4

6

0

20

CO

250

300

0

25

LVET

500

1000

0

20

dpdt

78

79

80

0

50

PFT

0

.

7

0

.

8

0

.

9

0

200

RFV

100

150

0

20

SBP

a

60

80

0

20

DBP

a

80

100

0

20

MAP

a

50

100

0

20

PP

a

100

150

0

20

SBP

b

60

80

0

20

DBP

b

80

100

0

20

MBP

b

50

100

0

20

PP

b

1

.

0

1

.

5

2

.

0

0

25

PP

amp

0

50

0

25

AP

0

50

0

20

AIx

50

100

150

0

20

Tr

5

10

15

0

20

PWV

a

5

10

15

0

25

PWV

cf

10

15

0

20

PWV

br

5

10

15

0

20

PWV

fa

35

40

45

0

20

dia

asca

25

30

0

20

dia

dta

12

.

5

15

.

0

17

.

5

0

20

dia

abda

8

10

0

20

dia

car

80

100

0

20

2

4

0

20

drop

fin

5

10

0

20

drop

ankle

100

200

300

0

20

SVR

20

40

60

80

0

25

age

60

80

0

20

HR

40

60

80

0

25

SV

4

6

0

25

CO

250

300

0

20

LVET

500

1000

0

25

dpdt

78

79

80

0

250

PFT

0

.

7

0

.

8

0

.

9

0

50

RFV

100

150

0

20

SBP

a

60

80

0

25

DBP

a

80

100

0

25

MAP

a

50

100

0

20

PP

a

100

150

0

20

SBP

b

60

80

0

25

DBP

b

80

100

0

20

MBP

b

50

100

0

25

PP

b

1

.

0

1

.

5

2

.

0

0

20

PP

amp

0

50

0

20

AP

0

50

0

25

AIx

50

100

150

0

25

Tr

5

10

15

0

25

PWV

a

5

10

15

0

25

PWV

cf

10

15

0

20

PWV

br

5

10

15

0

25

PWV

fa

35

40

45

0

20

dia

asca

25

30

0

25

dia

dta

12

.

5

15

.

0

17

.

5

0

20

dia

abda

8

10

0

25

dia

car

80

100

0

20

2

4

0

50

drop

fin

5

10

0

25

drop

ankle

100

200

300

0

25

SVR

(a) Information level 1

20

40

60

80

0

20

age

60

80

0

20

HR

40

60

80

0

20

SV

4

6

0

20

CO

250

300

0

20

LVET

500

1000

0

25

dpdt

78

79

80

0

20

PFT

0

.

7

0

.

8

0

.

9

0

100

RFV

100

150

0

20

SBP

a

60

80

0

20

DBP

a

80

100

0

20

MAP

a

50

100

0

25

PP

a

100

150

0

20

SBP

b

60

80

0

20

DBP

b

80

100

0

20

MBP

b

50

100

0

50

PP

b

1

.

0

1

.

5

2

.

0

0

25

PP

amp

0

50

0

20

AP

0

50

0

20

AIx

50

100

150

0

20

Tr

5

10

15

0

25

PWV

a

5

10

15

0

20

PWV

cf

10

15

0

20

PWV

br

5

10

15

0

20

PWV

fa

35

40

45

0

20

dia

asca

25

30

0

20

dia

dta

12

.

5

15

.

0

17

.

5

0

20

dia

abda

8

10

0

20

dia

car

80

100

0

20

2

4

0

20

drop

fin

5

10

0

20

drop

ankle

100

200

300

0

25

SVR

(b) Information level 2

20

40

60

80

0

20

age

60

80

0

20

HR

40

60

80

0

50

SV

4

6

0

25

CO

250

300

0

20

LVET

500

1000

0

20

dpdt

78

79

80

0

20

PFT

0

.

7

0

.

8

0

.

9

0

100

RFV

100

150

0

20

SBP

a

60

80

0

20

DBP

a

80

100

0

20

MAP

a

50

100

0

20

PP

a

100

150

0

20

SBP

b

60

80

0

20

DBP

b

80

100

0

20

MBP

b

50

100

0

20

PP

b

1

.

0

1

.

5

2

.

0

0

25

PP

amp

0

50

0

25

AP

0

50

0

25

AIx

50

100

150

0

20

Tr

5

10

15

0

20

PWV

a

5

10

15

0

20

PWV

cf

10

15

0

25

PWV

br

5

10

15

0

25

PWV

fa

35

40

45

0

20

dia

asca

25

30

0

20

dia

dta

12

.

5

15

.

0

17

.

5

0

20

dia

abda

8

10

0

20

dia

car

80

100

0

20

2

4

0

25

drop

fin

5

10

0

20

drop

ankle

100

200

300

0

20

SVR

(c) Information level 3

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

120

140

Pressure [mmHg]

Aortic Root

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

120

140

Carotid

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

120

140

Pressure [mmHg]

Thoracic Aorta

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

120

140

Brachial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

120

140

Pressure [mmHg]

Radial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

120

140

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

120

140

Pressure [mmHg]

Iliac Bifurcation

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

120

140

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

120

140

Pressure [mmHg]

Femoral

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

120

140

Anterior Tibial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

120

140

Pressure [mmHg]

Sup. Middle Cerebral

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

120

140

Sup. Temporal

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

Time (s)

120

140

Pressure [mmHg]

Digital

True Value

Prediction (Mean)

Prediction (Std Dev)

Prediction (Min/Max)

(a) Information level 1

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Pressure [mmHg]

Aortic Root

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Carotid

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

120

140

Pressure [mmHg]

Thoracic Aorta

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Brachial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

120

140

Pressure [mmHg]

Radial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Pressure [mmHg]

Iliac Bifurcation

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Pressure [mmHg]

Femoral

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Anterior Tibial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Pressure [mmHg]

Sup. Middle Cerebral

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Sup. Temporal

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

Time (s)

100

120

140

Pressure [mmHg]

Digital

True Value

Prediction (Mean)

Prediction (Std Dev)

Prediction (Min/Max)

(b) Information level 2

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Aortic Root

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Carotid

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Thoracic Aorta

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Brachial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Radial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Iliac Bifurcation

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Femoral

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Anterior Tibial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Sup. Middle Cerebral

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Sup. Temporal

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

Time (s)

100

150

Pressure [mmHg]

Digital

True Value

Prediction (Mean)

Prediction (Std Dev)

Prediction (Min/Max)

(c) Information level 3

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

100

120

140

Pressure [mmHg]

Aortic Root

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

100

125

150

Carotid

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

120

140

Pressure [mmHg]

Thoracic Aorta

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

100

120

140

Brachial

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

120

140

Pressure [mmHg]

Radial

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

100

120

140

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

100

120

140

Pressure [mmHg]

Iliac Bifurcation

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

100

120

140

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

100

120

140

Pressure [mmHg]

Femoral

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

100

120

140

Anterior Tibial

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

120

140

Pressure [mmHg]

Sup. Middle Cerebral

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

120

140

Sup. Temporal

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

Time (s)

120

140

Pressure [mmHg]

Digital

True Value

Prediction (Mean)

Prediction (Std Dev)

Prediction (Min/Max)

(a) Information level 1

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Aortic Root

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Carotid

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Thoracic Aorta

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Brachial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Radial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Iliac Bifurcation

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Femoral

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Anterior Tibial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Sup. Middle Cerebral

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Sup. Temporal

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

Time (s)

100

150

Pressure [mmHg]

Digital

True Value

Prediction (Mean)

Prediction (Std Dev)

Prediction (Min/Max)

(b) Information level 2

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Aortic Root

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Carotid

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Pressure [mmHg]

Thoracic Aorta

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Brachial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Pressure [mmHg]

Radial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Iliac Bifurcation

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Femoral

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Anterior Tibial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Pressure [mmHg]

Sup. Middle Cerebral

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Sup. Temporal

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

Time (s)

100

125

150

Pressure [mmHg]

Digital

True Value

Prediction (Mean)

Prediction (Std Dev)

Prediction (Min/Max)

(c) Information level 3

Test Case 1

Test Case 2

Test Case 3

0

.

025

0

.

050

0

.

075

0

.

100

0

.

125

CRPS

True Parameters

Test Case 1

Test Case 2

Test Case 3

0

5

10

15

Relative L1 Error

(a) CRPS on the inverse problem.

(b)

L

1

error on the forward and unified problem.

Aortic

Root

Carotid

Thoracic

Aorta

Brachial

Radial

Abdominal

Aorta

Iliac

Bifurcation

Common

Iliac

Femoral

Anterior

Tibial

Sup. Middle

Cerebral

Sup.

Temporal

Digital

0

.

05

0

.

10

0

.

15

0

.

20

0

.

25

0

.

30

0

.

35

0

.

40

Relative L1 Error

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Aortic Root

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Carotid

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Pressure [mmHg]

Thoracic Aorta

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Brachial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Pressure [mmHg]

Radial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Iliac Bifurcation

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Pressure [mmHg]

Femoral

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

150

Anterior Tibial

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Pressure [mmHg]

Sup. Middle Cerebral

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

100

125

150

Sup. Temporal

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

Time (s)

100

125

150

Pressure [mmHg]

Digital

True Value

Prediction

RFV

80

PPa

100

PPb

100

100150

PWVfa

10

15

10

SVR

8

100

0.5

0

品

0.025

0.000

0.02

0.00

0.025

000°0

0.2

0.0

0.5

0.0

10°0

00°0

08

SV

PFT

100

MBP

100

AIx

50

PWVbr

15

oppet

17.5

15.0

10

0℃

80

80

0

10

0.05

00°0

20

+0

84

0.05

0.00

0.05

0.00

0.02

00'0

0.2

0.0

0.2

0.2

0.0

15

5.0

H

apdp

1000

DBPa

80

DBP

08

AP

PWVef

10

piPei

ufdop

500

09

09

2.5

0

7'0

0.0

0.002

0.000

0.05

0.00

0.05

0.00

0.05

0.00

0.2

0.0

0.2

0.0

9'0

0.0

150

150

2.0

45

age

LVET

SBPa

SBPb

PWVa

10

osvep

40

100

1.5

100

100

0.1

0.0+

z0°0

0.00

0.02

00°0

0000

0.2

+0°0

0.0

Z0

08

20

40

60

80

0

5

age

60

80

0

.

0

2

.

5

HR

40

60

80

0

.

0

2

.

5

SV

4

6

0

.

0

2

.

5

CO

250

300

0

5

LVET

500

1000

0

5

dpdt

78

79

80

0

5

PFT

0

.

7

0

.

8

0

.

9

0

5

RFV

100

150

0

5

SBP

a

60

80

0

5

DBP

a

80

100

0

5

MAP

a

50

100

0

5

PP

a

100

150

0

.

0

2

.

5

SBP

b

60

80

0

5

DBP

b

80

100

0

5

MBP

b

50

100

0

5

PP

b

1

.

0

1

.

5

2

.

0

0

5

PP

amp

0

50

0

.

0

2

.

5

AP

0

50

0

5

AIx

50

100

150

0

5

Tr

5

10

15

0

5

PWV

a

5

10

15

0

.

0

2

.

5

PWV

cf

10

15

0

5

PWV

br

5

10

15

0

5

PWV

fa

35

40

45

0

5

dia

asca

25

30

0

.

0

2

.

5

dia

dta

12

.

5

15

.

0

17

.

5

0

5

dia

abda

8

10

0

5

dia

car

80

100

0

5

2

4

0

5

drop

fin

5

10

0

5

drop

ankle

100

200

300

0

.

0

2

.

5

SVR

Figure A.9: PWP, inverse problem: InVAErt predictions over parameters, the median sample with full input information (level 1). InVAErt is particularly susceptible to posterior collapse, i.e., the predictions over the parameters become overconfident and the uncertainty bounds fail to capture the true parameter.

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

75

100

Pressure [mmHg]

Aortic Root

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

75

100

125

Carotid

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

80

100

Pressure [mmHg]

Thoracic Aorta

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

75

100

125

Brachial

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

80

100

Pressure [mmHg]

Radial

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

75

100

125

75

100

125

Pressure [mmHg]

Iliac Bifurcation

75

100

125

(a) Age: 25 (blue) to 65 (red)

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

75

100

Pressure [mmHg]

Aortic Root

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

60

80

100

Pressure [mmHg]

Thoracic Aorta

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

75

100

Pressure [mmHg]

Radial

75

100

125

Pressure [mmHg]

Iliac Bifurcation

(b) DBP: 55 (blue) to 90 (red)

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

Time (s)

75

100

Pressure [mmHg]

Aortic Root

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

Time (s)

75

100

125

Carotid

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

Time (s)

80

100

Pressure [mmHg]

Thoracic Aorta

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

Time (s)

75

100

125

Brachial

100

Radial

125

(c) HR: 60 (blue) to 90 (red)

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

75

100

125

Pressure [mmHg]

Aortic Root

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

80

100

120

Pressure [mmHg]

Thoracic Aorta

100

120

Radial

(d) SBP: 80 (blue) to 165 (red)

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

75

100

125

Pressure [mmHg]

Aortic Root

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

75

100

125

Carotid

0

.

0

0

.

2

0

.

4

0

.

6

Time (s)

75

100

125

Pressure [mmHg]

Aortic Root

Pressure [mmHg]

Pressure [mmHg]

Pressure [mmHg]

Pressure [mmHg]

g]

80

100

125

80

100

125

75

100

75

0

0

0

0

.

.

.

.

0

0

0

0

0

0

.

.

2

2

0

0

.

.

4

0

Time (s)

.

6

0

.

8

1

.

0

Superficial Middle Cerebral

0

.

4

0

.

6

Time (s)

0

.

4

Time (s)

4

0

Femoral

Iliac Bifurcation

0

0

.

.

6

8

1

.

0

75

100

125

80

100

125

75

100

0

0

0

0

.

.

.

.

0

0

0

0

Pressure [mmHg]

0

.

2

Pressure [mmHg]

Pressure [mmHg]

Pressure [mmHg]

g]

0

.

2

sure [mmHg]

75

100

0

125

.

2

75

100

125

0

.

2

75

100

0

0

0

.

.

4

0

0

Time (s)

0

0

.

0

0

.

0

0

.

4

Time (s)

0

.

4

Time (s)

.

4

0

0

.

.

.

6

0

.

8

0

.

4

Time (s)

1

.

0

0

.

6

Superficial Middle Cerebral

0

.

2

Superficial Temporal

0

.

0

.

6

2

0

.

4

0

.

6

Anterior Tibial

Time (s)

0

.

4

.

6

0

Time (s)

.

8

0

.

4

Femoral

0

Iliac Bifurcation

0

.

.

6

6

ssure [mmHg]

0

0

.

.

2

2

.

6

0

6

0

.

.

2

2

0

1

.

0

125

100

75

125

100

125

75

100

75

125

100

125

75

100

125

75

100

75

125

100

125

75

100

125

75

100

75

125

100

125

75

100

125

75

100

75

100

125

75

100

125

75

100

0

0

0

0

0

0

0

0

0

0

0

0

0

0

.

.

.

.

.

.

.

.

.

.

.

.

.

.

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

.

.

.

.

.

.

.

.

.

.

.

.

.

.

2

2

2

2

2

2

2

2

2

2

2

2

2

2

0

.

4

Time (s)

0

.

4

Time (s)

0

.

4

Time (s)

0

.

4

Time (s)

0

.

4

Time (s)

0

.

4

Time (s)

0

.

4

Time (s)

0

.

4

Time (s)

0

.

4

Time (s)

0

.

4

Time (s)

0

.

4

Time (s)

0

.

4

Time (s)

0

.

4

Time (s)

0

.

4

S

0

.

02

0

.

04

0

.

06

0

.

08

0

.

10

0

.

12

0

.

14

CRPS

0

.

0

2

.

5

5

.

0

7

.

5

10

.

0

12

.

5

15

.

0

17

.

5

Relative L1 Error

Case 1

Case 2

Case 3

A.4 Atmospheric Cold Bubble

In atmospheric modelling, large-eddy simulations (LES) help to understand the turbulent dynamics within convective storms [39] that are usually observed only by scalar local measurements of wind (anemometers), other scalar measurements obtained at surface-level weather stations (e.g., pressure, humidity), and, if available, three-dimensional wind lidar records. A particularly destructive phenomenon associated with storms are downbursts , sudden local downdrafts that cause extreme winds when hitting the surface. Manual tuning of a numerical model is usually required to fit a simulation to match an observed downburst (e.g., [40]), due to the model complexity involving several parameterizations such as cloud microphysics. The model can then be configured to explore the sensitivity of the extreme winds both to the numerical model parameters and to the physical environmental conditions [38]. In addition to improving the scientific understanding and forecasts for more precise warnings, simulations of downbursts help to design robust infrastructure, such as wind turbines. However, the amount of simulations performed for model tuning (inverse problem) and sensitivity analysis (forward problem) is limited by the high computational cost to typically tens to hundreds of simulations. The affordable number of numerical simulations is either used for strategic expert tuning or to train an emulator for the parameter-to-data map, which is cheaper to evaluate and can then be used for in-depth sensitivity analysis (e.g., Gaussian Processes [9]; quadratic regression [36]). We would like to highlight that the presented test case is conceptually similar to downbursts, but far from simulating real-world scenarios due to its idealized nature and reduced complexity. However, we believe that test cases like this one will help to pave the way for handling realistic cases with methodologies such as FUSE.

Model Description The numerical data for the cold bubble simulation is obtained with the PyCLES model [42]. It solves the anelastic Navier-Stokes equations as formulated by [41], evolving around a hydrostatic reference state,

with reference profiles α 0 ( z ) for specific volume and p 0 ( z ) for pressure, and ( x 1 , x 2 , x 3 ) = ( x, y, z ) and ( u 1 , u 2 , u 3 ) = ( u, v, w ) for simplicity. The anelastic equations of motion are given by

encompassing the momentum equations (a), the entropy equation (b), and the continuity equation (c). We use standard conventions for summing and the Kronecker delta δ ij , and left out the terms and further equations referring to the Coriolis force and moisture, as they do not apply to the small-scale and dry cold bubble simulation. The buoyancy term b depends on the specific volume, moisture, gravity, and the reference pressure profile p 0 , p ′ is the dynamic pressure perturbation over the hydrostatic reference state, and Σ i is an additional momentum source term, which is expected to be small for the cold bubble experiment. In the entropy equation, Q is the diabatic heating rate, T is the temperature, and ˙ S is a source term of irreversible entropy sources. Finally, τ ij and γ s,i are the sub-grid scale (SGS) stresses acting on velocity and entropy, which model the effect of turbulence smaller than the grid scale in a diffusive way as

relating the sub-grid stress to the strain rate S ij = 1 / 2( ∂ i u j + ∂ j u i ) of the resolved flow. The eddy viscosity ν t may be related to the diffusivity D t for heat and other scalars ϕ , and ultimately to the corresponding SGS stresses γ ϕ,i by the turbulent Prandtl number Pr t by

For three-dimensional simulations, one may choose a sub-grid scale parameterization for ν t such as a Smagorinsky-type first-order closure or a higher-order closure based on a prognostic equation for

turbulent kinetic energy. For the two-dimensional cold bubble case, however, we stick to uniform values for both ν t and D t , independent of the turbulent Prandtl number.

The domain of size ( L x , L z ) = (51 . 6 km , 6 . 4 km) exhibits a neutrally stratified background profile ( ∂ z θ = 0 , where θ denotes potential temperature), and no background winds. An elliptic cold air anomaly is prescribed by

where default values and ranges for all parameters can be found in Table 9, with the resulting flow fields shown in Figure A.12.

Numerical Setup The PyCLES model by [42] relies on a WENO finite volumes scheme that exhibits exceptional stability, implemented in Cython. It is used here in the setup described by the authors, at resolution ∆ x = ∆ z = 50 mand an adaptive time stepping bound to ∆ t ≤ 5 s in order to record time series measurements at a temporal resolution of five seconds.

The data is recorded at the eight locations marked in Figure A.12, at horizontal locations x = 15 , 20 km (about 10 and 5 km from the center of the anomaly), and vertical heights of z = 50 , 100 , 250 , 500 , and 2000 m, each being part of the numerical grid.

In terms of computational complexity, a PyCLES simulation in the given setup takes about half an hour on eight cores.

4.0

t=0 s

4.0

t=0 s

4.0

t=0 s

[x]

2.0

5

10

2.0

5

10

2.0

5

10

1-4

6-9

1-4

6-9

1-4

6-9

0.0+

0.0 +

0.0+

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

4.0

4.0

4.0

1

N

2.0

2.0

2.0

0.0+

0.0+

0.0+

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

t=600 s

t=600 s

t=600 s

4.0

4.0

4.0

[x] :

N

2.0

2.0

2.0

0.0+

0.0 +

0.0 +

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

4.0

t=900s

4.0

t=900 s

4.0

t=900s

[ux]

N

2.0

2.0

2.0

0.0+

0.0

+

0.0

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

x [km]

x [km]

x [km]

-20

0

20

-10

-5

0

-20

-10

0

10

20

u [m/s]

temperature anmoaly [K]

w[m/s]

4.0

t=0.0 s

4.0

t=0.0 s

4.0

t=0.0 s

1

2.0

5

10

2.0

5

10

2.0

5

10

1-4

6-9

1-4

6-9

1-4

6-9

0.0

0.0+

0.0+

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

4.0

t=300.0 s

4.0

t=300.0 s

4.0

t=300.0s

1

2.0

√

2.0

2.0

0.0+

0.0 +

0.0+

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

t=600.0s

t=600.0 s

t=600.0 s

4.0

4.0

4.0

[x]

2.0

2.0

2.0

N

0.0+

0.0 +

0.0+

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

4.0 -

t=900.0s

4.0

t=900.0 s

4.0

t=900.0 s

[x]

2.0

2.0

2.0

N

0.0

0.0 +

0.0 +

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

10.0

15.0

20.0

25.0

x [km]

x [km]

x [km]

-40

20

0

20

40

-20

-15

-10

-5

0

20

-10

0

10

20

u [m/s]

temperature anmoaly [K]

w[m/s]

99

.

7 %

95

.

5 %

68

.

3 %

p

φ

(

θ

|

x

∗

)

30

60

viscosity

8

16

24

amplitude

2

.

5

5

.

0

7

.

5

x

r

2

.

4

3

.

0

3

.

6

z

c

30

60

diffusivity

1

.

6

2

.

4

z

r

30

60

viscosity

8

16

24

amplitude

2

.

5

5

.

0

7

.

5

x

r

2

.

4

3

.

0

3

.

6

z

c

1

.

6

2

.

4

z

r

(a) Parameter estimates from the test sample with largest (worst) CRPS.

99

.

7 %

95

.

5 %

68

.

3 %

p

φ

(

θ

|

x

∗

)

30

60

viscosity

8

16

24

amplitude

2

.

5

5

.

0

7

.

5

x

r

2

.

4

3

.

0

3

.

6

z

c

0

30

60

diffusivity

1

.

6

2

.

4

z

r

30

60

viscosity

8

16

24

amplitude

2

.

5

5

.

0

7

.

5

x

r

2

.

4

3

.

0

3

.

6

z

c

1

.

6

2

.

4

z

r

(b) Parameter estimates from the test sample with median CRPS.

0

200

400

600

800

Time (s)

-

60

-

40

-

20

0

u [m/s]

Location 1

True

Prediction

0

200

400

600

800

Time (s)

-

2

0

2

4

w [m/s]

Location 1

0

200

400

600

800

Time (s)

-

60

-

40

-

20

0

u [m/s]

Location 3

0

200

400

600

800

Time (s)

-

10

0

10

w [m/s]

Location 3

0

200

400

600

800

Time (s)

-

40

-

20

0

u [m/s]

Location 6

0

200

400

600

800

Time (s)

-

5

0

5

w [m/s]

Location 6

0

200

400

600

800

Time (s)

-

40

-

20

0

u [m/s]

Location 8

0

200

400

600

800

Time (s)

-

10

0

10

w [m/s]

Location 8

(a) Sample with greatest (worst) error of FUSE.

0

200

400

600

800

Time (s)

-

15

-

10

-

5

0

u [m/s]

Location 1

True

Prediction

0

200

400

600

800

Time (s)

0

1

2

3

w [m/s]

Location 1

0

200

400

600

800

Time (s)

-

15

-

10

-

5

0

u [m/s]

Location 3

0

200

400

600

800

Time (s)

0

2

4

6

w [m/s]

Location 3

0

200

400

600

800

Time (s)

-

30

-

20

-

10

0

u [m/s]

Location 6

0

200

400

600

800

Time (s)

-

2

0

2

4

w [m/s]

Location 6

0

200

400

600

800

Time (s)

-

30

-

20

-

10

0

u [m/s]

Location 8

0

200

400

600

800

Time (s)

-

5

0

5

w [m/s]

Location 8

(b) Sample with median error of FUSE.

0

200

400

600

800

Time (s)

-

40

-

20

0

u [m/s]

Location 1

True

Prediction

0

200

400

600

800

Time (s)

-

2

0

2

4

w [m/s]

Location 1

0

200

400

600

800

Time (s)

-

40

-

20

0

u [m/s]

Location 3

0

200

400

600

800

Time (s)

-

5

0

5

10

w [m/s]

Location 3

0

200

400

600

800

Time (s)

-

40

-

30

-

20

-

10

0

u [m/s]

Location 6

0

200

400

600

800

Time (s)

0

2

4

w [m/s]

Location 6

0

200

400

600

800

Time (s)

-

40

-

30

-

20

-

10

0

u [m/s]

Location 8

0

200

400

600

800

Time (s)

-

5

0

5

10

w [m/s]

Location 8

(c) Sample with strong velocity gradients (Fig. A.13).

25

50

75

0

20

40

diffusivity

25

50

75

0

20

40

viscosity

10

20

0

20

40

amplitude

2

.

5

5

.

0

7

.

5

0

20

40

x

r

2

.

5 3

.

0 3

.

5

0

20

z

c

1

2

0

20

40

z

r

(a) Sample on which FUSE has the median CRPS.

25

50

75

0

20

40

diffusivity

25

50

75

0

20

viscosity

10

20

0

20

40

amplitude

2

.

5

5

.

0

7

.

5

0

20

x

r

2

.

5 3

.

0 3

.

5

0

20

z

c

1

2

0

20

z

r

(b) Sample on which FUSE has worst CRPS.

0

200

400

600

800

Time (s)

-

20

-

10

0

u [m/s]

Loc. 1: (15km, 50m)

0

200

400

600

800

Time (s)

-

2

0

2

4

w [m/s]

Loc. 1: (15km, 50m)

3

4

5

6

7

x

r

(a)

x

r

0

200

400

600

800

Time (s)

-

40

-

20

0

u [m/s]

Loc. 1: (15km, 50m)

0

200

400

600

800

Time (s)

-

2

0

2

4

w [m/s]

Loc. 1: (15km, 50m)

7

.

5

10

.

0

12

.

5

15

.

0

17

.

5

20

.

0

22

.

5

amplitude

(b) Amplitude

0

200

400

600

800

Time (s)

-

20

-

10

0

u [m/s]

Loc. 1: (15km, 50m)

0

200

400

600

800

Time (s)

-

2

0

2

4

w [m/s]

Loc. 1: (15km, 50m)

1

.

2

1

.

4

1

.

6

1

.

8

2

.

0

2

.

2

2

.

4

z

r

(c)

z

r

0

200

400

600

800

Time (s)

-

20

-

15

-

10

-

5

0

u [m/s]

Loc. 1: (15km, 50m)

0

200

400

600

800

Time (s)

0

2

4

w [m/s]

Loc. 1: (15km, 50m)

2

.

6

2

.

8

3

.

0

3

.

2

3

.

4

z

c

(d)

z

c

0

200

400

600

800

Time (s)

-

20

-

15

-

10

-

5

0

u [m/s]

Loc. 1: (15km, 50m)

0

200

400

600

800

Time (s)

0

2

4

w [m/s]

Loc. 1: (15km, 50m)

10

20

30

40

50

60

70

viscosity

(e) Viscosity

0

200

400

600

800

Time (s)

-

20

-

15

-

10

-

5

0

u [m/s]

Loc. 1: (15km, 50m)

0

200

400

600

800

Time (s)

-

2

0

2

4

w [m/s]

Loc. 1: (15km, 50m)

10

20

30

40

50

60

70

diffusivity

(f) Diffusivity

(a) Location 1 (

x

= 15

km,

z

= 50

m)

(b) Location 5 (

x

= 20

km,

z

= 50

m)

30.0

0.20

30.0

0.20

umaz [m/s]: relative error

umax [m/s]: relative error

25.0

0.15

25.0

0.15

20.0

amplitude

20.0

0.10

0.10

15.0

15.0

0.05

0.05

10.0

10.0

5.0+

0.00

5.0

0.00

0.0

25.0

50.0

75.0

2.0

4.0

6.0

8.0

0.0

25.0

50.0

2.0

4.0

6.0

8.0

diffusivity

diffusivity

Zr

30.0

30.0

umax [m/s): FUSE-truth

umax [m/s]: FUSE-truth

25.0

25.0

4

amplitude

20.0

2

amplitude

20.0

2

0

0

15.0

15.0

2

-2

10.0

-4

F00T

5.0

-6

5.0 +

-6

0.0

25.0

50.0

75.0

2.0

4.0

6.0

8.0

0.0

25.0

50.0

T5.0

2.0

4.0

6.0

8.

diffusivity

diffusivity

Zr

30.0

60

30.0

60

25.0

50

25.0

50

amplitude

20.0

40

[s/u]zoun

amplitude

20.0

40

[s/u]

30

2Dun

15.0

15.0

20

20

F001

10.0

10

10

5.0 +

+09

0.0

25.0

50.0

75.0

2.0

4.0

6.0

0.0

25.0

50.0

75.0

2.0

4.0

6.0

diffusivity

Zr

difusivity

Cr

30.0

60

30.0

60

25.0

50

25.0+

09

amplitude

20.0

40

[s/u] zrun

amplitude

20.0

40

[s/u]

30

30

15.0

15.0

20

20

10.0

001

10

10

5.0

5.0

0.0

25.0

50.0

75.0

2.0

4.0

6.0

0.0

25.0

50.0

75.0

2.0

4.0

6.0

diffusivity

Zr

diffusivity

Zr

[m/s]

u

8.0

6.0

4.0

2.0

x

r

25.0

18.0

12.0

5.0

amplitude

3.5

3.2

2.8

2.5

z

c

2.5

2.0

1.5

1.0

z

r

0.0

25.0

50.0

75.0

diffusivity

75.0

50.0

25.0

0.0

viscosity

2.0

4.0

6.0

8.0

x

r

5.0

12.0

18.0

25.0

amplitude

2.5

2.8

3.2

3.5

z

c

1.0

1.5

2.0

2.5

z

r

10

20

30

40

50

60

u

[m/s]

(a) FUSE model.

2.0

4.0

6.0

8.0

x

r

5.0

12.0

18.0

25.0

amplitude

2.5

2.8

3.2

3.5

z

c

1.0

1.5

2.0

2.5

z

r

0.0

25.0

50.0

75.0

diffusivity

0.0

25.0

50.0

75.0

viscosity

2.0

4.0

6.0

8.0

x

r

5.0

12.0

18.0

25.0

amplitude

2.5

2.8

3.2

3.5

z

c

1.0

1.5

2.0

2.5

z

r

10

20

30

40

50

60

(b) Numerical ground truth.

.

.

.

.

.

.

.

[m/s]

u

2.0

4.0

6.0

8.0

x

r

5.0

12.0

18.0

25.0

amplitude

2.5

2.8

3.2

3.5

z

c

1.0

1.5

2.0

2.5

z

r

0.0

25.0

50.0

75.0

diffusivity

0.0

25.0

50.0

75.0

viscosity

2.0

4.0

6.0

8.0

x

r

5.0

12.0

18.0

25.0

amplitude

2.5

2.8

3.2

3.5

z

c

1.0

1.5

2.0

2.5

z

r

0

.

0

2

.

5

5

.

0

7

.

5

10

.

0

12

.

5

15

.

0

17

.

5

20

.

0

u

[m/s]

(a) FUSE model.

2.0

4.0

6.0

8.0

x

r

5.0

12.0

18.0

25.0

amplitude

2.5

2.8

3.2

3.5

z

c

1.0

1.5

2.0

2.5

z

r

0.0

25.0

50.0

75.0

diffusivity

0.0

25.0

50.0

75.0

viscosity

2.0

4.0

6.0

8.0

x

r

5.0

12.0

18.0

25.0

amplitude

2.5

2.8

3.2

3.5

z

c

1.0

1.5

2.0

2.5

z

r

0

.

0

2

.

5

5

.

0

7

.

5

10

.

0

12

.

5

15

.

0

17

.

5

20

.

0

(b) Numerical ground truth.

1: (15km, 50m)

u

w

2: (15km, 100m)

u

w

3: (15km, 250m)

u

w

4: (15km, 500m)

u

w

5: (15km, 2km)

u

w

6: (20km, 50m)

u

w

7: (20km, 100m)

u

w

8: (20km, 250m)

u

w

9: (20km, 500m)

u

w

10: (20km, 2km)

u

w

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

Relative L1 Loss

(a) Forward problem: Error between the prediction of s based on the true parameters ξ ∗ and the true output time series s .

1: (15km, 50m)

u

w

2: (15km, 100m)

u

w

3: (15km, 250m)

u

w

4: (15km, 500m)

u

w

5: (15km, 2km)

u

w

6: (20km, 50m)

u

w

7: (20km, 100m)

u

w

8: (20km, 250m)

u

w

9: (20km, 500m)

u

w

10: (20km, 2km)

u

w

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

Relative L1 Loss

(b) Unified prediction: Error between the predicted ensemble mean ¯ s based on the input time series u , and the true output time series s .

A.5 Properties of the Total Variation Metric

Consider the total variation distance d ( µ, µ ∗ ), where µ and µ ∗ are measures over u ∈ U , which is defined as

Further, consider a random vector ξ ∈ Ξ ⊆ R m , the function h : Ξ → U , and the probability measures ρ and ρ ∗ over Ξ . Since u = h ( ξ ) , the measures µ and µ ∗ can be considered as the pushforward measures induced by ρ through the function h , µ = h # ρ and µ ∗ = h # ρ ∗ . We assume all measures admit densities and use the same notation for measures µ ( A ) , A ⊆ U , and their densities µ ( u ) , u ∈ U , since the distinction is clear from the arguments.

We consider the conditional probability measures ρ ( ξ | u ) , ρ ∗ ( ξ | u ) . Then, µ and µ ∗ are given by:

and the measure of set A is:

Assume u is uniquely determined by ξ through the map u = h ( ξ ) and that ρ ( ξ | u ) is hence highly concentrated at ξ = h -1 ( u ) and therefore:

where δ is the Dirac delta function and

The definition of the total variation is then written as:

Therefore due to the unique definition of u from ξ via h , the two definitions of the total variation are equivalent. The relation between the total variation of the original and the conditional measures is then given by:

A.6 Detailed Derivation of the Forward Objective

We start with d ( ˜ G θ # µ , ˜ G # µ ) and our goal is to bound this metric by the supervised operator learning objective. For a function u , its corresponding images ˜ G ( u ) and ˜ G θ ( u ) , and for any set A ∈ B ( S ) , where B is a Borel σ -algebra over S :

where w A ( s ) is the indicator function that is one if s ∈ A and zero otherwise. Let B ′ ( U ) be the σ -algebra on U generated by the pre-images of sets in B ( S ) under ˜ G . Then,

Here, ∥ · ∥ L 1 ( Y ) is the L 1 norm over the space of function S . We are interested in parametric PDEs where we have access to parameters ξ ∈ Ξ ⊂ R m , the PDE parameters, sampled from ρ , a probability measure on R m . Operator learning involves a map between continuous functions, so an intermediate step needs to be considered to first lift the parameters ξ to a continuous space before performing the action of the operator. For this purpose, we can, for example, consider h to be the band-limited lifting described in the main text, composed using a lifting l : R m → B Ω and an inverse Fourier Transform F -1 : B Ω →U . In this formulation, l maps the parameters to k to the Paley-Wiener space of band-limited functions in Fourier space, B Ω = { l ∈ L 2 ( R ) : supp l ⊂ [ -Ω , Ω] } , thus truncating the frequencies that can be used to represent the data. We consider µ = h # ρ ( ξ ) , the pushforward of ρ ( ξ ) under h , and use a change of variables:

for any integrable function f . We can apply this property to:

We can define the operators G : Ξ →S and G θ : Ξ →S and write:

Finally, we arrive at

A.7 Detailed Derivation of the Inverse Objective

In the supervised operator learning problem, we have access to samples u of functions but we do not have access to the distribution that generates u . Assume that ˜ G θ is Lipschitz continuous:

Assuming that µ ϕ is an approximation of µ , we can relate these two measures by the total variation of their respective pushforward measures under the forward surrogate G θ using the continuity properties of the operator (see Appendix A.5),

for some constant C ∈ R , which we generally assume to be equal to one. This property holds because of the continuity of ˜ G θ and because the total variation is preserved under continuous transformations. We assume that the functions u are generated by solving a parametric PDE, where the parametric uncertainty over ξ is propagated onto u through the likelihood µ ( u | ξ ) . Performing parameter estimation corresponds to sampling from a conditional distribution ρ ϕ ( ξ | u ) approximating the posterior ρ ( ξ | u ) . To do so, we first bound the previous distance by the total variation between ρ ϕ ( ξ | u ) and ρ ( ξ | u ) over Ξ (see Appendix A.5):

Combining the two inequalities, we get

which means that we can minimize the right hand side of the inequality (29).

A.8 Formulation of the FMPE loss

For our implementation of Flow-Matching Posterior Estimation (FMPE), we closely follow [60]. In the main text (Eqn. 5), we formulated the FMPE objective as

for the total variation distance d on measures over the parameter space Ξ ⊆ R m . We will minimize this distance in order to approximate the underlying distribution, ρ ϕ ≈ ρ , and sample M samples from ρ ϕ ( ξ | u ) with the help of FMPE. We also defined the transform (6),

which transforms an infinite-dimensional input u ∈ U to a finite-dimensional conditional information ˆ u ∈ R k in Fourier space.

The FMPE method learns an invertible transform between the target distribution of parameters, conditional on the additional information, and a standard normal distribution. This transform is modelled as an ordinary differential equation with artificial time dimension t ∈ [0 , 1] and solution ψ t, ˆ u : R m → R m as

where ξ 0 follows a standard normal distribution. Trajectories in the artificial time t can be obtained as ξ t = ψ t, ˆ u ( ξ ) . This way, the target probability ρ ( ξ | ˆ u ) is approximated as

by solving the transport equation d dt ρ t + div ( ρ t v t, ˆ u ) = 0 .

During training, ψ 0 , ˆ u i ( ξ i ) is fit to resemble a standard normal on the training samples { ξ i , ˆ u i } n i =1 . During evaluation, a standard normal sample ξ 0 is drawn, and transformed to a sample that approximately follows ρ ( ξ | u ) as

In order to fully describe ψ t, ˆ u , it hence suffices to learn the trajectory velocity function v t, ˆ u . Let p t ( ξ | ξ 1 ) be a sample-conditional Gaussian probability path with vector field l t ( ξ t | ξ 1 ) , and let time be distributed as t ∼ p ( t ) . Then, [60] formulate the FMPE loss as

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The claims made in this paper are clearly highlighted in the abstract and introduction. The main contributions have been listed clearly in the introduction through bullet points. The results section is structured such that it justifies these claims.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Critical assumptions and practical limitations are presented in the Discussion section.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All assumptions and formulas are clearly stated, numbered, and crossreferenced in the paper with theoretical justification and detailed derivation provided in the appendix.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The experimental setup is described extensively, with model architectures and training details outlined in the appendix. Furthermore, we provide all code, data, and models (including baselines) required to reproduce the experiments outlined in this paper.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The data and code is provided in the supplementary material, uploaded with the paper submission.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Training details are described for the all experiments, with more extensive training details specified in the appendix and Supplementary Material. Additionally, all the selected model configurations are provided in the code.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Errors or scores are reported by the mean+/-standard deviation across all test samples. Furthermore, figures displaying box plots of these errors and scores are presented in the appendix. The quantities that these numbers and figures represent are clearly stated in the respective text and captions.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The amount of compute required for each of the experiments, as well as the total amount of compute hours for all experiments, is outlined in the training details reported in the appendix.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: This work complies with the NeuIPS code of ethics, with special attention to the data and model documentation, access to research artifacts, and disclosure of essential elements for reproducibility.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This work is primarily focused on a broader strategy to solve problems in scientific computing, i.e. solve systems of differential equations. Direct paths to negative applications, such as deepfakes or generation of misinformation are not present. Additionally, while this work presents experiments on synthetic cardiovascular data, it does not meet the NeurIPS guidelines to immediately cause negative societal impact without a prior evaluation protocol. For health data specifically, detailed evaluation protocols issued by public health and safety organizations must be considered to meet in-silico, in-vitro, and in-vivo evaluation standards.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The data used in these experiments has a very low risk for misuse. The dataset which is released with this work is based on fluid dynamics for turbulent systems, while the other dataset used in experiments is a publicly available, synthetic dataset of cardiovascular models.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The dataset for full-body haemodynamics is open source and appropriate cited and credited. The atmospheric turbulence dataset is produced by the authors, but based on an openly available code cited in the text. Baseline models used for comparison are properly cited in the text.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The code that reproduces the results is documented and easy to read. The documentation is contained in the actual code.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a new approach, FUSE to the simultaneous learning of an emulator and statistical estimation of underlying \"discrete\" parameters in a joint training step.   \n\nThe approach splits the problem into two:   (i) the forward problem which is modelled through a FNO neural operator approach,  effectively learns a map from a space of finite-dimensional parameters to an output function.   (ii) The inverse step then seeks to learn a conditional distribution for the parameters $\\xi$ based on measurements of the continuous input $u$ using a flow-matching approach.    This yields two loss functions which are simultaneously optimised.\n\nThe authors present two challenging problems as test cases.\n\nSTRENGTHS:\nThe authors have identified an important problem which is genuinely challenging and have proposed a potential solution to this problem.   They have demonstrated the effectiveness of their approach on two challenging PDE-based problems.   The components of the proposed approach are not novel, but their combination seems to be a new approach, demonstrating originality.\n\nWEAKNESSES:\nI struggled at times to understand the actual methodology in the paper.   The two strategies for forward and inverse are relatively clear, but their combination is far from clear to me,  which leads me to question the broad applicability of this method.\n\nMy understanding of the methodology is that the authors require data of the form $(y, u, \\theta)$,  for which $u$ is a continuous input, $y$ is a continuous output and $\\theta$ is an intermediate set of parameters.     The forward model learns the deterministic map between $y$ and $\\theta$ and the inverse model learns the probabilistic relation between $\\theta$ and $u$.    If this is the case, then the need to propose a combined methodology seems unclear to me.     The fact that the two optimisation problems in (4) appear to be decoupled, suggests that this is the case.\n\nThe challenge that the authors describe is most relevant in settings where data from $\\theta$ or $u$ are not available, in which case we must rely on $y$ and $u$ to identify $\\theta$ while training the model,  but this seems to be not the case?  Otherwise, I am struggling to see the need for this combination of methods.\n\nI also think the authors could be more comprehensive in their literature review, which has been very narrowly focused on a handful of methods.  There are both more recent methods, and quite a history of older approaches which seek to address this problem in different contexts around science and engineering.\n\nQUESTIONS:\nThe limitations indicated above need to be clarified -- some clarity is needed in the introduction to spell out the applicability of this methodology.   If my understanding is correct, then stronger motivation of why a combined approach is even needed.\n\nDiscrete parameters is not really the right terminology -- which seems to suggest your data is ordinal valued (1,2,3).   I suspect you mean that your parameter is a finite dimensional vector rather than a function?\n\nFinally, the literature review seems very focused on neural network approaches, yet there are other approaches based on Gaussian processes etc.  Could the authors provide some relevant background which is more encompassing.\n\nLIMITATIONS:\nLimitations have been well addressed.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a framework to tackle simultaneously forward (simulation of the system) and inverse (estimation of key parameters of the system) problems for PDEs. Namely, the authors suppose the existence of an underlying parameter $\\xi$ that characterizes the input functions $u$ of the PDE, and therefore formulate a probabilistic framework where the output solution $s$ is sampled according to $\\xi \\sim p(\\xi | u)$ and $\\hat{s} = G^\\theta(\\xi)$. The authors employ a two-loss objective for training where the first one aims to train the operator and the second one is used to approximate the true distribution $p^*(\\xi | u)$. The method is tested on complex PDE systems, that depend on multiple parameters, namyely atmospheric cold bubble (ACB) and plus wave propagation (PWP). It obtains better performance than existing methods and convincing uncertainty propagation results.\n\nSTRENGTHS:\n* The paper proposes FUSE, a unifying approach for both inverse and forward problems for PDEs, with a relevant theoretical framework.\n* The method obtains SOTA results on complex PDEs on the forward metrics, and is very competitive in the inverse task. It outperforms all the baselines in the OOD regime, showcasing the robustness of the method.\n* The uncertainty propagation property is impressive, as shown in Figure 4 and 5.\n* They propose the first application of flow matching for PDE problems, and integrate Fourier layers in the architecture.\n\nWEAKNESSES:\n* The datasets used are quite complex as they include many parameters and several equations. It could be best to provide additional visualizations of the data and further describe the data format used by each block of the architecture to better illustrate the different components.\n* The paper does not justify the use of flow matching compared to a different probabilistic approach. Would there be a difference between DDPM and Flow matching in this case ?\n* There is a lack of details on flow matching, particularly on the parametrization $\\psi_{u, t}$ and its inverse $\\psi^{-1}_{u, t}$.\n\nQUESTIONS:\n* Do you have access to the true parameters $\\xi$ during training ?\n* Do you train the neural operators with parameters $\\xi$ sampled from the flow matching ?\n* Is the training done sequentially or in parallel ? According to the minimization objective, each loss only depends on a single set of parameters.\n* How many steps do you take at inference ?\n* Did you try other generative models other than flow matching ?\n* Did you try a fully probabilistic framework ? For instance assuming that the observations of $s$ were not deterministic given $\\xi$ ?\n* What is the training and inference time of the method ?\n\nLIMITATIONS:\nThere is a limitation section that discusses the assumptions of the method.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study the problem of joint prediction of continuous fields and statistical estimation of parameters for physical systems governed by PDEs.  Prior work had focused on operator learning and then inference to determine the statistical parameters. Here, the propose to solve for both jointly in their method FUSE which combines Neural Operators with Flow Matching Posterior Estimation (FMPE). The authors then test their method on important applied problems in haemodynamics and large eddy simulations showing advantages in both the inverse and surrogate problems.\n\nSTRENGTHS:\n- Good application domains studied including haemodyanmics and atmospheric large-eddy simulation of bubbles.\n- Interesting problem to study parametric PDEs\n- Identified proper limitation of numerical methods when the PDE parameters are not known exactly and calibration techniques must be used to learn the parameter from data in inverse problems.\n- Nice overview of Neural Operators\n- Nice to also mention ROMs and recent work in deep learning.\n- Good that the paper tackles both forward problems with UQ and inverse problems\n- CRPS is good uncertainty metric\n- Also good that OOD case is tested\n- Thorough an diverse evaluation\n- Results show that the proposed FUSE method is performing strongly\n- FUSE maintains the discretization invariance property of NOs\n\nWEAKNESSES:\n- Add references to the classical numerical methods, such as LeVeque in the introduction\n- Can add reference to the Multiwavelet Neural Operator, Gupta et al., NeurIPS 2021 in the introduction.\n- Missing literature references: Neural Operator methods with UQ for the Forward Model Simulation including Bayesian Neural Operator, Magnani et al., and see the overview and detailed comparison in Mouli et al., \"Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs\", ICML 2024, which studies sensitivity analysis and UQ for Neural Operators. I think these works should be cited and the methods compared to as baselines. Also see Ma et al., \"Calibrated Uncertainty Quantification for Operator Learning via Conformal Prediction\", https://arxiv.org/abs/2402.01960, 2024 for conformal prediction techniques.\n- The main limitation is lack of benchmarking against the above FNO + UQ baselines\n- More recent state-of-the-art baselines, e.g., diffusion models (DDPM, DDIM) and variants could also be compared.\n\nQUESTIONS:\n1, In parametric PDEs, are the authors discussing the PDE parameter or the BC/IC as mentioned in the introduction and why are they discrete? The authors should clarify this.\n2. How are the two loss functions weighted and counter-balanced? Is there a hyper-parameter to tune?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose \"FUSE\", a combination of multiple neural operator models, which are trained to jointly solve PDE forward problems and perform parameter inference of given parametric PDE.\nThe main idea is to start from a range of PDE solutions obtained from various parameter values, and then train neural operators that can interpolate both in parameter space and in solution space.\nThe approach is evaluated on two systems of parametric PDE, and compared to a range of similar neural operator and sampling approaches.\n\nSTRENGTHS:\nThe results of the computational experiments seem impressive, especially given that only a relatively small number of samples is used (O(1000)). The main idea of jointly learning the parameter inference and the forward PDE solution is valid and interesting. The comparison to a range of other approaches shows that the proposed approach works well.\n\nWEAKNESSES:\n1) Figure 1 is not very clear. The caption does not explain what is shown in the figure, the parameters, models, inputs and outputs are not mentioned.\n\n2) Many references are only referred to by their pre-print. The proper journal / conference should be given instead.\n\n3) Inference times are not provided, and no comparison to classical solvers is performed (even though they are being used to generate the training data). Also see question 4.\n\n4) The paper (main and appendix) does not contain a lot of details regarding implementation and applicability (Questions 5 and 6).\n\n\nMinor:\n1) l88: \"observatbles\" -> observables\n2) l110: G is now not an operator as defined in l87, but a function on a finite-dimensional space (with a range in a function space). The redefinition is confusing.\n\nQUESTIONS:\n1) l64: why is the framework called \"FUSE\"? Is it an acronym for something?\n\n2) l93: should it not be $\\mu$ instead of $\\mu^*$ on the right argument of d as well? Otherwise, how can we minimize the distance to the full measure, not just its approximation?\n\n3) l142: what does it mean that \"$\\mathcal{P}$ is a map that lifts the channels of the dimensions of the input function\"? What does \"lifting channels of dimensions\" mean?\n\n4) The authors state that (l31) \"iterative and thus expensive calibration procedures\" are required for classical solvers as a drawback, but then 180 GPU hours (l604) are required for training the proposed model. The authors do not comment on this, so my question: why is it beneficial to train, or \"calibrate\" the neural operator model for such a long time, as opposed to using the \"iterative and expensive\" calibration procedures for classical solvers?\n\n5) There are very little details on the \"learnable maps\" mentioned throughout the explanation of the approach. Which maps are used? Are all of these MLPs, deep, which nonlinearities? These details do not need to be mentioned in the main paper, but even the appendix does not contain them.\n\n6) There is very little detail on the the applicability of the approach. For which PDEs is it useful, when will it not work, how much data is needed for which types of problems, etc.? These questions do not need to be answered completely, but there is not even a simple example where this is being discussed or studied.\n\nLIMITATIONS:\nThe authors discuss limitations in terms of future work, which is fine. Negative societal impact is not discussed, where the authors argue that only PDE problems are being solved, without immediate concerns - which is questionable, especially because the authors use pulse propagation in the human cardiovascular system as an example where the approach could be used (and where humans may be harmed it it does not work in practice in a hospital).", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a novel framework called FUSE that unifies surrogate modeling and parameter identification for parametric partial differential equations (PDEs). Traditionally, field prediction and statistical estimation of discrete parameters have been separately handled by using operator learning surrogates and simulation-based inference, respectively. FUSE proposes a combined approach that aims to enhance the accuracy and robustness of both tasks.\n\nFUSE is designed to jointly predict continuous fields and infer the distributions of discrete parameters, leveraging a unified pre-training step to amortize the computational costs associated with both the inverse and surrogate models. It employs a probabilistic extension of operator learning through Flow Matching Posterior Estimation (FMPE) to effectively handle parametric uncertainties. This unified approach facilitates in-depth model calibration and supports sensitivity analysis and uncertainty quantification (UQ) at a significantly reduced computational cost.\n\nThe authors demonstrate the proposed methodology on two applications: pulse wave propagation (PWP) in the human arterial network and an atmospheric cold bubble (ACB) simulation. They emphasize the advantages of FUSE in both the inverse (parameter estimation) and forward (continuous field prediction) tasks.\n\nSTRENGTHS:\n1. Problem Definition: The authors have mathematically defined the statistical estimation of discrete parameters, a significant problem encountered in practical applications.\n\n2. Integrated Approach: FUSE provides a holistic and efficient solution for addressing parametric uncertainties in PDEs by integrating surrogate modeling and parameter inference tasks within a single framework.\n\n3. Robustness: The use of FMPE in a probabilistic setting makes the model enhance the robustness of model predictions under varying conditions and uncertainties.\n\nWEAKNESSES:\n1. Lack of Novelty: This paper appears to be an application of FMPE (Flow Matching Posterior Estimation) to the inverse problem, lacking significant novel developments.\n\n2. Uncommon Problems: The problems addressed in this paper have not been widely discussed within the machine learning community (though it might be due to my lack of familiarity).\nQuestions\n\nQUESTIONS:\n1. In the supplementary section (Fig A.1, A.2), it seems that the identified parameters can cover a very wide range. However, these parameters do not significantly impact the PDE solution (Fig A.3, A.4). Is it correct to interpret these variables as having low sensitivity?\n\n2. In the worst case of (Fig A.3, A.4), even when considering uncertainty (i.e., various possible parameter values), the obtained solution seems quite different from the true one. Should this be interpreted as indicating the presence of unknown physics? \n\nThese questions might arise from my lack of familiarity with the specific applications discussed, as mentioned in the weaknesses above.\n\n3. Can the methodology in this paper be applied if the boundary conditions are set as parameters?\n\nLIMITATIONS:\nThe authors have effectively explained the limitations of their approach in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-13"}