NOMAD: Nonlinear Manifold Decoders for Operator Learning
Anonymous Author(s)
Af ﬁ liation Address email
Abstract
Supervised learning in function spaces is an emerging area of machine learningresearch with applications to the prediction of complex physical systems such asﬂ uid ﬂ ows, solid mechanics, and climate modeling.
By directly learning maps(operators) between in ﬁ nite dimensional function spaces, these models are able tolearn discretization invariant representations of target functions.
A common approach is to represent such target functions as linear combinations of basis elementslearned from data.
However, there are simple scenarios where, even though thetarget functions form a low dimensional submanifold, a very large number of basiselements is needed for an accurate linear representation.
Here we present NOMAD,a novel operator learning framework with a nonlinear decoder map capable oflearning ﬁ nite dimensional representations of nonlinear submanifolds in functionspaces.
We show this method is able to accurately learn low dimensional representations of solution manifolds to partial differential equations while outperforminglinear models of larger size.
Additionally, we compare to state-of-the-art operatorlearning methods on a complex ﬂ uid dynamics benchmark and achieve competitiveperformance with a signi ﬁ cantly smaller model size and training cost.
Introduction
[Figure]
Machine learning techniques have been applied to great success for modeling functions between ﬁ nite dimensional vectorspaces.
For example, in computer vision(vectors of pixel values) and natural language processing (vectors of word embeddings) these methods have produced stateof-the-art results in image recognition [15]and translation tasks [42].
However, notall data has an obvious and faithful representation as ﬁ nite dimensional vectors.
Inparticular, functional data is mathematically represented as a vector in an in ﬁ nite dimensional vectorspace.
This kind of data appears naturally in problems coming from physics, where scenarios in ﬂ uiddynamics, solid mechanics, and kinematics are described by functions of continuous quantities.
Supervised learning in the in ﬁ nite dimensional setting can be considered for cases where we want to map functional inputs to target functional outputs.
For example, we might wish to predict the velocity of a ﬂ uid as function of time given an initial velocity ﬁ eld, or predict the pressure ﬁ eld across the surface of the Earth given temperature measurements.
This is similar to a ﬁ nite dimensional regression problem, except that we are now interested in learning an operator between spaces of functions.
We refer to this as a supervised operator learning problem : given a data-set of N pairsof functions { ( u 1 , s 1 ) , . . .
, ( u N , s N ) } , learn an operator F which maps input functions to outputfunctions such that F ( u i )
=
s i , ∀ i .
One approach to solve the supervised operator learning problem is to introduce a parameterizedoperator architecture and train it to minimize a loss between the model’s predicted functions andthe true target functions in the training set.
One of the ﬁ rst operator network architectures waspresented in [6] with accompanying universal approximation guarantees in the uniform norm.
Theseresults were adapted to deep networks in [26] and led to the DeepONet architecture and its variants[45, 28, 17].
The Neural Operator architecture, motivated by the composition of linear and nonlinearlayers in neural networks, was proposed in [23].
Using the Fourier convolution theorem to computethe integral transform in Neural Operators led to the Fourier Neural Operator [24].
Other recentarchitectures include approaches based on PCA-based representations [1], random feature approaches[31], wavelet approximations to integral transforms [13], and attention-based architectures [19].
A common feature shared among many of these approaches is that they aim to approximate anoperator using three maps: an encoder, an approximator, and a decoder, see Figure 1 and Section 3for more details.
In all existing approaches embracing this structure, the decoder is constructed as alinear map.
In doing so, the set of target functions is being approximated with a ﬁ nite dimensionallinear subspace in the ambient target function space.
Under this setting, the universal approximationtheorems of [6, 20, 21] guarantee that there exists a linear subspace of a large enough dimensionwhich approximates the target functions to any prescribed accuracy.
However, as with ﬁ nite dimensional data, there are scenarios where the target functional dataconcentrates on a low dimensional nonlinear submanifold.
We refer to the phenomenon of data infunction spaces concentrating on low dimensional submanifolds as the Operator Learning Manifold
Hypothesis , see Figure 1.
For example, it is known that certain classes of parametric partial differentialequations admit low dimensional nonlinear manifolds of solution functions [7].
Although linearrepresentations can be guaranteed to approximate these spaces, their required dimension can becomevery large and thus inef ﬁ cient in capturing the true low dimensional structure of the data.
In this paper, we are motivated by the Operator Learning Manifold Hypothesis to formulate a newclass of operator learning architectures with nonlinear decoders.
Our key contributions can besummarized as follows.
• Limitations of Linear Decoders: We describe in detail the shortcomings of operator
learning methods with linear decoders and present some fundamental lower bounds along with an illustrative operator learning problem which is subject to these limitations.
• Nonlinear Manifold Decoders (NOMAD):
This motivates a novel operator learning frame- work with a nonlinear decoder that can ﬁ nd low dimensional representations for ﬁ nitedimensional nonlinear submanifolds in function spaces.
• Enhanced Dimensionality Reduction: A collection of numerical experiments involving linear transport and nonlinear wave propagation shows that, by learning nonlinear submanifolds of target functions, we can build models that achieve state-of-the-art accuracy whilerequiring a signi ﬁ cantly smaller number of latent dimensions.
• Enhanced Computational Ef ﬁ ciency: As a consequence, the resulting architectures con-
tain a signi ﬁ cantly smaller number of trainable parameters and their training cost is greatly reduced compared to competing linear approaches.
We begin our presentation in Section 2 by providing a taxonomy of representative works in theliterature.
In Section 3 we formally de ﬁ ne the supervised operator learning problem and discussexisting approximation strategies, with a focus on highlighting open challenges and limitations.
In
Section 4 we present the main contributions of this work and illustrate their utility through the lensof a pedagogical example.
In Section 5 we provide a comprehensive collection of experimentsthat demonstrate the performance of using NOMAD against competing state-of-the-art methods foroperator learning.
Section 6 summarizes our main ﬁ ndings and discusses lingering limitations andbroader impact.
Additional details on architectures, hyperparameter selection, and training details areprovided in the Supplemental Materials.
Related
Low Dimensional Representations in Finite Dimensional Vector Spaces
: Finding low dimensional representations of high dimensional data has a long history, going back to 1901 with theoriginal formulation of principal components analysis (PCA) [33].
PCA is a linear method that worksbest when data concentrates on low dimensional subspaces.
When data instead concentrates onlow dimensional nonlinear spaces, kernelized PCA [36] and manifold learning techniques such as
Isomap and diffusion maps [40, 9] can be effective in ﬁ nding nonlinear low dimensional structure,see [41] for a review.
The recent popularity of deep learning has introduced new methods for ﬁ ndinglow dimensional structure in high dimensional data-sets, most notably using auto-encoders [46, 4]and deep generative models [10, 18].
Relevant to our work, such techniques have found success inapproximating submanifolds in vector spaces corresponding to discretized solutions of parametricpartial differential equations (PDEs) [37, 35, 12], where a particular need for nonlinear dimensionreduction arises in advection-dominated problems common to ﬂ uid mechanics and climate science[22, 29].
Low Dimensional Representations in In ﬁ nite Dimensional Vector Spaces: The principles behind PCA generalize in a straightforward way to functions residing in low dimensional subspacesof in ﬁ nite dimensional Hilbert spaces [44].
In the ﬁ eld of reduced order modeling of PDEs this issometimes referred to as proper orthogonal decomposition [5] (see [25] for an interesting expositionof the discrete version and connections to the Karhunen-Loève decomposition).
Af ﬁ ne representationsof solution manifolds to parametric PDEs and guarantees on when they are effective using the notionof linear n -widths [34] have been explored in [7].
As in the case of ﬁ nite dimensional data, usinga kernel to create a feature representation of a set of functions, and then performing PCA in theassociated Reproducing Kernel Hilbert Space can give nonlinear low dimensional representations[39].
The theory behind optimal nonlinear low dimensional representations for sets of functions isstill being developed, but there has been work towards de ﬁ ning what “optimal” should mean in thiscontext and how it relates to more familiar geometric quantities [8].
Operator Learning
Notation: Let us ﬁ rst set up some notation and give a formal statement of the supervised operatorlearning problem.
We de ﬁ ne C ( X ; R d ) as the set of continuous functions from a set X to R d .
When
X ⊂ R n , we de ﬁ ne the Hilbert space,
[Equation]
This is an in ﬁ nite dimensional vector space equipped with the inner product � f, g � =
� X f ( x ) g ( x ) dx .
When X is compact, we have that C ( X ; R d ) ⊂
L 2 ( X ; R d ) .
We now can present a formal statementof the supervised operator learning problem.
Problem Formulation: Suppose we are given a training data-set of N pairs of functions ( u i , s i ) ,where u i ∈ C
( X ; R d u ) with compact X ⊂ R d x , and s
i ∈ C
( Y ; R d s ) with compact Y
⊂ R d y .
Assume there is a ground truth operator G : C ( X ; R d u ) →
C ( Y ; R d s ) such that G ( u i ) =
s i andthat the u
i are sampled i.i.d.
from a probability measure on C ( X ; R d u ) .
The goal of the supervisedoperator learning problem is to learn a continuous operator F : C ( X ; R d x ) → C ( Y ; R d s ) toapproximate G .
To do so, we will attempt to minimize the following empirical risk over a class ofoperators F θ , with parameters θ ∈ Θ
⊂ R d θ ,
[Equation]
An Approximation Framework for Operators: A popular approach to learning an operator
G : L 2 ( X ) → L 2 ( Y ) acting on a probability measure µ on L 2 ( X ) is to construct an approximationout of three maps [21] (see Figure 1),
[Equation]
The ﬁ rst map, E : L 2 ( X ) → R m is known as the encoder.
It takes an input function and mapsit to a ﬁ nite dimensional feature representation.
For example, E could take a continuous functionto its point-wise evaluations along a collection of m sensors, or project a function onto m basisfunctions.
The next map A : R m → R n is known as the approximation map.
This can be interpretedas a ﬁ nite dimensional approximation of the action of the operator G .
Finally, the image of theapproximation map is used to create the output functions in L 2 ( Y ) by means of the decoding map
D : R n → L 2 ( Y ) .
We will refer to the dimension, n , of the domain of the decoder as the latentdimension .
The composition of these maps can be visualized in the following diagram.
[Equation]
Linear Decoders: Many successful operator learning architectures such as the DeepONet [26],the (pseudo-spectral) Fourier Neural Operator in [20], LOCA [19], and the PCA-based method in[1] all use linear decoding maps D .
A linear D can be de ﬁ ned by a set of functions τ
i ∈
L 2 ( Y ) , 142 i = 1 , . .
. , n , and acts on a vector β ∈ R n as
[Equation]
For example, the functions τ i can be built using trigonometric polynomials as in the Ψ -FNO [20], beparameterized by a neural network as in DeepONet [26], or created as the normalized output of akernel integral transform as in LOCA [19].
Limitations of Linear Decoders: We can measure the approximation accuracy of the operator Fwith two different norms.
First is the L 2 ( µ ) operator norm,
[Equation]
Note that the empirical risk used to train a model for the supervised operator learning problem (see(1)) is a Monte Carlo approximation of the above population loss.
The other option to measure theapproximation accuracy is the uniform operator norm.
For a compact set U ⊂ L 2 ( X ) we can de ﬁ nethe error
[Equation]
When a linear decoder is used for F = D ◦ A ◦ E , a data-dependent lower bound to each of these 153 errors can be derived.
L 2 lower bound: When the pushforward measure has a ﬁ nite second moment, its covarianceoperator Γ : L 2 ( Y ) → L 2 ( Y ) is self-adjoint, positive semi-de ﬁ nite, and trace-class, and thus admitsan orthogonal set of eigenfunctions spanning its image, { φ 1 , φ 2 , . . . } with associated decreasingeigenvalues λ 1 ≥ λ 2 ≥ ... .
The decay of these eigenvalues indicates the extent to which samplesfrom G # µ concentrate along the leading ﬁ nite-dimensional eigenspaces.
It was shown in [21] that forany choice of E and A , these eigenvalues give a fundamental lower bound to the expected squared L 2error of the operator learning problem with architectures as in (3) using a linear decoder D ,
[Equation]
This result can be further re ﬁ ned to show that the optimal choice of functions τ i (see equation (4))
fora linear decoder are given by the leading n eigenfunctions of the covariance operator { φ 1 , . . . , φ n } .
The interpretation of this result is that the best way to approximate samples from G # µ with ann -dimensional subspace is to use the subspace spanned by the ﬁ rst n “principal components” of theprobability measure G # µ .
The error incurred by using this subspace is determined by the remainingprincipal components, namely the sum of their eigenvalues � k>n λ k .
The operator learning literature slowly decaying pushforward covariance spectrum (such assolutions to advection-dominated PDEs) these lower bounds cause poor performance for models ofthe form (3)
Uniform lower bound:
In the reduced order modelling of PDEs literature [7, 8, 22] there existsa related notion for measuring the degree to an n -dimensional subspace can approximate a set offunctions S ⊂
L 2 ( Y ) .
This is known as the Kolmogorov n -width [34], and for a compact set S isde ﬁ ned as
[Equation]
This measure of how well a set of functions can be approximated by a linear subspace in the uniformnorm leads naturally to a lower bound for the uniform error (6).
To see this, ﬁ rst note that for anyu ∈ U , the error from F ( u ) to G ( u ) is bounded by the minimum distance from G ( u ) to the image of
F .
For a linear decoder D : R n → L 2 ( Y ) , de ﬁ ne the (at most) n -dimensional V n = im( D ) ⊂
L 2 ( Y ) .
Note that im
( F ) ⊆
V n , and we may write
[Equation]
Taking the supremum of both sides over u ∈ U , and then the in ﬁ mum of both sides over all 180 n -dimensional subspaces V n gives
[Equation]
The quantity on the right is exactly the Kolmogorov n -width of G ( U ) .
We have thus proved thefollowing complementary statement to (7) when the error is measured in the uniform norm.
Proposition 1 Let U ∈
L 2 ( X ) be compact and consider an operator learning architecture as in (3) ,where D : R n → L 2 ( Y ) is a linear decoder.
Then, for any E : L 2 ( X ) → R p and A : R p → R n , the 185 uniform norm error of
F := D ◦ A ◦ E satis ﬁ es the lower bound
[Equation]
Therefore, we see that in both the L 2 ( µ ) and uniform norm, the error for an operator learning problemwith a linear decoder is fundamentally limited by the extent to which the space of output functions“ ﬁ ts” inside a ﬁ nite dimensional linear subspace.
In the next section we will alleviate this fundamentalrestriction by allowing decoders that can learn nonlinear embeddings of R n into L 2 ( Y ) .
Nonlinear Decoders for Operator Learning
A Motivating Example: Consider the problem of learning the antiderivative operator mappingfunctions to their ﬁ rst-order derivative
[Equation]
acting on a set of input functions
[Equation]
[Equation]
The set of output functions is given by G ( U ) = { sin(2 π tx ) | 0 < t 0 < t < T } .
This is a onedimensional curve of functions in L 2 ([0 , 1]) parameterized by a single number t .
However, we wouldnot be able to represent this set of functions with a one-dimensional linear subspace.
In Figure 2bwe perform PCA on the functions in this set evaluated on a uniform grid of values of t .
We seethat the ﬁ rst 20 eigenvalues are nonzero and relatively constant, suggesting that an operator learningarchitecture with a linear or af ﬁ ne decoder would need a latent dimension of at least 20 to effectivelyapproximate functions from G ( U ) .
Figure 2a gives a visualization of this curve of functions projectedonto the ﬁ rst three PCA components.
We will return to this example in Section 5, and see that anarchitecture with a nonlinear decoder can in fact approximate the target output functions with superioraccuracy compared to the linear case, using a single latent dimension that can capture the underlyingnonlinear manifold structure.
[Figure]
Operator Learning Manifold Hypothesis: We now describe an assumption under which a nonlinear decoder is expected to be effective, and use this to formulate the NOMAD architecture.
To thisend, let µ be a probability measure on L 2 ( X ) and G : L 2 ( X ) → L 2 ( Y ) .
We assume that there existsan n -dimensional manifold M ⊆
L 2 ( Y ) and an open subset
O ⊂ M such that
[Equation]
In connection with the manifold hypothesis in deep learning [3, 2], we refer to this as the Operator
Learning Manifold Hypothesis .
There are scenarios where it is known this assumption holds, such asin learning solutions to parametric PDEs [29].
This assumption motivates the construction of a nonlinear decoder for the architecture in (3) asfollows.
For each u , choose v ( u ) ∈
O such that
[Equation]
[Equation] [Figure] [Equation] [Figure]
D
[Equation]
[Figure]
Let φ : O → R n be a coordinate chart for
O ⊂ M .
We can represent v ( u ) ∈ O by its 216 coordinates φ ( v ( u ))
∈ R n . Consider a choiceof encoding and approximation maps such that
A ( E ( u )) gives the coordinates for v ( u ) .
If thedecoder were chosen as D := φ − 1 then by construction, the operator F :=
D ◦ A ◦ E will 221 satisfy
[Equation]
Therefore, we interpret a learned decoding mapas attempting to give a ﬁ nite dimensional coordinate system for the solution manifold.
Considera generalized decoder of the following form
[Equation]
This induces a map from D : R n → L 2 ( Y ) ,as D ( β ) =
˜ D ( β , · ) .
If the solution manifold
M is a ﬁ nite dimensional linear subspace in 229 L 2 (
Y ) spanned by { τ i } ni =1 , we would want a 230 decoder to use the coef ﬁ cients along the basisas a coordinate system for M .
A generalized decoder could learn this basis as the output of a deepneural network to act as
[Equation]
However, if the solution manifold is not linear, then we should learn a nonlinear coordinate systemgiven by a nonlinear D .
A nonlinear version of ˜ D can be parameterized by using a deep neuralnetwork f
: R n × Y → R which jointly takes as arguments ( β , y ) ,
[Equation]
When used in the context of an operator learning architecture of the form (3), we call a nonlineardecoder from (17) NOMAD (NOnlinear MAnifold Decoder).
Figure 3 presents a visual comparisonbetween linear and nonlinear decoders.
Summary of NOMAD: Under the assumption of the Operator Learning Manifold Hypothesis , wehave proposed a fully nonlinear decoder (17) to represent target functions using architectures of theform (3).
We next show that using a decoder of the form (17) results in operator learning architectureswhich can learn nonlinear low dimensional solution manifolds.
Additionally, we will see that whenthese solution manifolds do not “ ﬁ t” inside low dimensional linear subspaces, architectures withlinear decoders will either fail or require a signi ﬁ cantly larger number of latent dimensions.
Results
In this section we investigate the effect of using a linear versus nonlinear decoders as building blocksof operator learning architecture taking the form (3).
In all cases, we will use an encoder E whichtakes point-wise evaluations of the input functions, and an approximator map A given by a deepneural network.
The linear decoder parametrizes a set of basis functions that are learned as the outputsof an MLP network.
In this case, the resulting architecture exactly corresponds to the DeepONetmodel from [26].
We will compare this against using NOMAD where the nonlinear decoder is builtusing an MLP network that takes as inputs the concatenation of β ∈ R n and a given query pointy ∈ Y .
All models are trained with by performing stochastic gradient descent on the loss function in(1).
The reported errors are measured in the relative L 2 ( Y ) norm by averaging over all functionalpairs in the testing data-set.
More details about architectures, hyperparameters settings, and trainingdetails are provided in the Supplemental Materials.
Learning the Antiderivative Operator: First, we revisit the motivating example from Section4, where the goal is to learn the antiderivative operator (10) acting on the set of functions (11).
In
Figure 2c we see the performance of a model with a linear decoder and NOMAD over a range oflatent dimensions n .
For each choice of n , 10 experiments with random initialization seeds wereperformed, and the mean and standard deviation of testing errors are reported.
We see that
the
NOMAD architecture consistently outperforms the linear one (by one order of magnitude), and caneven achieve a 10% relative prediction error using only n = 1 .
Solution Operator of a Parametric Advection PDE: Here we consider the problem of learningthe solution operator to a PDE describing the transport of a scalar ﬁ eld with conserved energy,
[Equation]
over a domain ( x, t ) ∈ [0 , 2] × [0 , 1] .
The solution operator maps an initial condition s ( x, 0)
=
u ( x )to the solution at all times s ( x, t ) which satis ﬁ es (18).
We consider a training data-set of initialconditions taking the form of radial basis functions with a very small ﬁ xed lengthscale centered atrandomly chosen locations in the interval [0 , 1] .
We create the output functions by evolving theseinitial conditions forward in time for 1 time unit according to the advection equation (18) (see
Supplemental Materials for more details).
Figure 4a gives an illustration of one such solution plottedover the space-time domain.
Performing PCA on the solution functions generated by these initial conditions shows a very slowdecay of eigenvalues (see Figure 4b), suggesting that methods with linear decoders will requirea moderately large number of latent dimensions.
However, since the data-set was constructed byevolving a set of functions with a single degree of freedom (the center of the initial conditions), wewould expect the output functions to form a solution manifold of very low dimension.
In Figure 4c we compare the performance of a linear decoder and NOMAD as a function of the latentdimension n .
Linear decoders yield poor performance for small values of n , while NOMAD appearsto immediately discover a good approximation to the true solution manifold.
[Figure]
Propagation of Free-surface Waves:
As a more challenging benchmark we consider the shallowwater equations; a set of hyperbolic equations that describe the ﬂ ow below a pressure surface in aﬂ uid [43].
The underlying PDE system takes the form
[Equation]
where ρ ( x, y, t ) the ﬂ uid height from the free surface, g is the gravity acceleration, and v 1 ( x, y, t ) ,v 2 ( x, y, t ) denote the horizontal and vertical ﬂ uid velocities, respectively.
We consider re ﬂ ectiveboundary conditions and random initial conditions corresponding to a random droplet falling intoa still ﬂ uid bed (see Supplemental Materials).
In Figure 5a we show the average testing error of amodel with a linear and nonlinear decoder as a function of the latent dimension.
Figure 5b showssnapshots of the predicted surface height function on top of a plot of the errors to the ground truth forthe best, worst, median, and a random sample from the testing data-set.
We additionally use this example to compare the performance of a model with a linear decoder and NOMAD to other state-of-the-art operator learning architectures.
For this experiment we includea harmonic feature expansion in the DeepONet, as it was shown in [28] that this can signi ﬁ cantlyimprove its performance.
We do not include this in the NOMAD architecture for this experiment(see Supplemental Materials for more details).
In Table 1, we present the mean relative error and itsstandard deviation for different operator learning methods, as well as the prediction that provides theworst error in the testing data-set when compared against the ground truth solution.
For each methodwe also report the number of its trainable parameters, the number of its latent dimension n , and thetraining wall-clock time in minutes.
Since the general form of the FNO [24] does not neatly ﬁ t intothe architecture given by (3), there is not a directly comparable measure of latent dimension for it.
We also observe that, although the model with NOMAD closely matches the performance of LOCA[19], its required latent dimension, total number of trainable parameters, and total training time areall signi ﬁ cantly smaller.
Discussion
Summary: We have presented a novel framework for supervised learning in function spaces.
Theproposed methods aim to address challenging scenarios where the manifold of target functions haslow dimensional structure, but is embedded nonlinearly into its associated function space.
Suchcases commonly arise across diverse functional observables in the physical and engineering sciences(e.g. turbulent ﬂ uid ﬂ ows, plasma physics, chemical reactions), and pose a signi ﬁ cant challengeto the application of most existing operator learning methods that rely on linear decoding maps,forcing them to require an excessively large number of latent dimensions to accurately representtarget functions.
To address this shortcoming we proposed a fully nonlinear framework that caneffectively learn low dimensional representations of nonlinear embeddings in function spaces, anddemonstrated that it achieves competitive accuracy to state-of-the-art operator learning methods while
[Figure]
[Table]
using a signi ﬁ cantly smaller number of latent dimensions [Table] leading to lighter model parametrizationsand reduced training cost.
Limitations: Our proposed approach relies on the Operator Learning Manifold Hypothesis (seeequation (12)), suggesting that cases where a low dimensional manifold structure does not exist willbe hard to tackle (e.g. target function manifolds with fractal structure, solutions to evolution equationswith strange attractors).
Moreover, even when the manifold hypothesis holds, the underlying effectivelatent embedding dimension is typically not known a-priori, and may only be precisely found viacross-validation.
Another direct consequence of replacing linear decoders with fully nonlinear mapsis that the lower bound in (9) needs to be rephrased in terms of a nonlinear n -width, which in generalcan be dif ﬁ cult to quantify.
Finally, in this work we restricted ourselves to exploring simple nonlineardecoder architectures such as an MLPs with the latent parameters β and query location y concatenatedas inputs.
Further investigation is needed to quantify the improvements that could be brought byconsidering more contemporary deep learning architectures, such as hypernetworks [14] which cande ﬁ ne input dependent weights for complicated decoder architectures.
One example of this idea inthe context of reduced order modeling can be found in Pan et.
al.
[32], where the authors propose ahypernetwork based method combined with a Implicit Neural Representation network [38].
Potential Negative Societal Impacts: Our contributions enables advances in accelerating themodeling, simulation, and optimization of physical and engineering systems.
As with any tool thatfurthers our understanding and ability to predict the outcomes of complex systems, there may beill-intentioned use cases, but we do not expect any speci ﬁ c negative impact from this work.