Physics-Driven ML-Based Modelling for Correcting Inverse Estimation
Authors: Ruiyuan Kang
Tingting Mu
Panos Liatsis
Dimitrios Kyritsis
Venue: NeurIPS 2023 spotlight (2023)
Keywords: Failure detection
Physical evaluation
Network-based optimization
Generative model
Hybrid surrogate model

Abstract: When deploying machine learning  estimators in science and engineering (SAE) domains, it is critical  to avoid failed estimations that can have disastrous consequences, e.g., in aero engine design. This work focuses on detecting and correcting  failed  state estimations before adopting them in SAE inverse problems, by  utilizing simulations and performance metrics guided by physical laws. We suggest to flag a machine learning estimation when its physical model error exceeds a feasible threshold, and propose a novel approach, GEESE, to correct it  through optimization, aiming at delivering both low error and high efficiency. The key designs of GEESE include (1) a hybrid surrogate error model to  provide fast  error estimations  to reduce simulation cost and to enable gradient based backpropagation of error feedback, and (2) two generative models to approximate the probability distributions of the candidate states for simulating the  exploitation and exploration behaviours. All three models are constructed as neural networks. GEESE is tested on three real-world SAE inverse problems and compared to a number of state-of-the-art optimization/search approaches. Results show that it fails the least number of times in terms of finding a feasible state correction, and requires physical evaluations less frequently in general.

Physics-Driven ML-Based Modelling for Correcting Inverse Estimation
Ruiyuan Kang" Bayanat AI Abu Dhabi, UAE ruiyuan. kang@bayanat.ai
Tingting Mu University of Manchester Manchester, M13 9PL, United Kingdom tingting.o mu@manchester ac. uk
Panos Liatsis
Dimitrios C. Kyritsis
Khalifa Univeristy Abu Dhabi, UAB
Khalifa Univeristy Abu Dhabi, UAE
panos wiw.tsiskku.... ac ae
dimitrios. kyritsiskkt.kkkeee.. ac.
Abstract
When deploying machine learning estimators in science and engineering (SAE) domains, it critical to avoid failed estimations that can have disastrous conse- quences, e.g., in aero engine design. This work focuses on detecting and correcting failed state estimations before adopting them SAE inverse problems, by utilizing simulations and performance metrics guided by physical laws. We suggest to flag a machine learning estimation when its physical model error exceeds a feasible threshold, and propose novel approach, GEESE, to correct through optimization, aiming at delivering both low error and high efficiency. The key designs of GEESE include (1) hybrid surrogate егror model to provide fast error estimations to reduce simulation cost and to enable gradient based backpropagation of error feed- back, and (2) two generative models to approximate the probability distributions the candidate states for simulating the exploitation and exploration behaviours. All three models are constructed neural networks. GEESE is tested three real-world SAE inverse problems and compared to a number of state-of-the-arr optimization/searc approaches. Results show that it fails the least number of times in terms of finding a feasible state correction, and requires physical evaluations less frequently in general.
Introduction
Many estimation problems in science and engineering (SAE) are fundamentally inverse problem, where the goal is estimate the state X' of system from its observation y y. Examples include estimating the temperature state from the observed spectrum in combustion diagnostics [I]. and discovering design parameters (state) of aero engine according parameters (observation) (2). Traditional physics-driven inverse solvers are supported by rigorous physical laws, which vary depending on the application, e.g., the two-colour method for spectrum estimation 3 and cycle analysis for aero engine design 4 Recent advances take advantage of machine learning (ML) techniques, constructing mapping functions F directly estimate the state from the observation, i.e., & = F(y) [5] 6 Z]. Such ML solutions are more straightforward to develop, moreover, efficient and easy use. However, ML-based state estimates can sometimes be erroneous, while SAE applications have very low error tolerance. One can imagine the disastrous
group of performance
'The work done during Ruiyuan's PhD and Postdoc at Khalifa Univeristy
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

consequences of providing unqualified aero engine design parameters. Therefore, it is critical to detect and correct failed ML estimations before adopting them.
This leads to special SAE requirement of evaluating the estimation correctness in the deployment process of an MI. estimator. Since the ground truth state is unknown this stage, indirect evaluation has to be performed. Such evaluations can be based on physical forward models and performance metrics 88 9 A common practice is combine multiple evaluations to obtain an accumulated physical error, enforcing quality control from different aspects.
When the physical error exceeds feasibility threshold, one has to remediate the concerned ML estimation. One practice for finding a better estimation is to directly minimize the physical error in state space [TO]. This requires solving black-box optimization problem, for which it is challenging to find its global optimum, iterative approaches are used to find a near-optimal solution [1 12 In each iteration, set of states selected to collect their physical errors, then error feedback is used to generate better state(s) until a near-optimal state is found. Physical error collection involves time-consuming simulations [3] [4], c.g., spectrum simulation which, despite taking just several minutes for each run [15, can become costly if queried many times. Consequently, the optimization process becomes time-consuming. Therefore, in addition to searching satisfactory state with as small as possible physical error, it is also vital to decrease the query times to the physical evaluation.
Our work herein is focused on developing an efficient algorithm for remediating the concerned ML estimation in deployment. We propose a novel correction algorithm, Generative Exploitation and Exploration guided by hybrid Surrogate Error (GEESE), building upon black-box optimization. It aims at finding a qualified state within an error tolerance threshold after querying the physical evaluations few times as possible. The key design elements of GEESE include: (1) A hybrid surrogate error model, which comprises an ensemble of multiple base neural networks, to provide fast estimation of the physical error and to enable informative gradient-based backpropagation of error feedback in model training. (2) A generative twin state selection approach, which consists of two generative neural networks for characterizing the distributions of candidate states, to effectively simulate the exploitation and exploration behaviours. We conduct thorough experiments to test the proposed algorithm and compare it with a series of state-of-the-ue art optimization/searee techniques. based on three real-work inverse problems. Results show that, among the compared methods, GEESE able to find a qualified state after failing the least number of times and needing to query the physical evaluations less times.
Related Work
Optimization in SAE: Development of SAE solutions often requires to formulate and solve opti- They are often black-box optimization due to the SAE nature. For
mization problems [66 instance, when the objective function is characterized through physical evaluations and solving partial differential equations (PDEs) [9), it is not given in closed form. Typical black-box optinization techniques include Bayesian Optimization 20). Genetic Algorithm (GA) (21). and Particle Swarm Optimization (PSO) 222 etc. They often require a massive number of queries to the objective func- tion in order to infer search directions for finding a near-optimal solution, which is time-consuming and expensive in SAE applications.
17 18
Instead, differentiable objective functions are constructed, and the problem is reduced to standard optimization, referred to as white-box optimization to be in contrast with black-box. A rich amount of well established solvers are developed for this, e.g., utilizing first-order and second-order gradient information 23 Some recent developments use neural networks to optimize differentiable physical model evaluations, c.g., Optnet (24) and iterative neural networks [25). However, physics-driven objective functions cannot always be formulated in differential form, e.g., errors evaluated by the physical forward model in acro engine simulation, which is a mixture of database data, map information and PDEs [26]. A grey-box setting is thus more suitable in practice, where one does not overwrap the evaluations as a black box or oversimplify them as a white box, but a mixture of both.
Surrogate Model in Black-box Optimization: To reduce the cost of querying objective function values in black-box optimization, recent approaches construct surrogate models to obtain efficient and cheap estimation of the objective function. This practice has been used in SAE optimization, where the objective functions are mostly based on physical evaluations. The most popular technique for constructing surrogate models is machine learning (ML), including neural networks and Gaussian pro-

---

cess models [27 28 29 The associated surrogate model is then incorporated within an optimization process, guided by. for instance, GA and Bayesian optimization, which generate states and interact
with it 30 29 or neural networks that work with differentiable surrogate models 31
32
12]
To
avoid overfitting, recent effort has been invested to develop surrogate models consistent with some pre-collected data, aiming at obtaining more reliable near-optimal solutions [3
34
35|
3d
37.
Nevertheless, there is no guarantee that a surrogate model can well approximate a physical model consistently. Indeed, this is the motivation for the proposed method, where surrogate models are used to speed up the querying process, while the decision in regards to the suitability of the solution is based on the actual physical evaluation.
Reinforcement Learning for Inverse Problems: In addition to black-box optimization based approaches, Reinforcement Learning (RL) [38 39] serves as an altemnatie framework for solving inverse problems 440 42 In an RI.-based solution framework, physical evaluations are wrapped as a black-box environment outputting scalar reward, and the actions are the states to estimate according to the observation. The behaviour of the environment simulated by training a world/critic model 43 44). which is equivalent to surrogate model of the physical evaluations. Different from black-box optimization based approaches, RL does not intend to search a feasible state estimation to provide state estimations, while the policy training is guided by optimizing an accumulated scalar reward or error Because of the desire of training& powerful policy model and the statistical nature of the reward, RL often requires many physical evaluations to collect diverse samples and validate training performance 49 [0]. This can be time-consuming when there is limited computing resource.
for the given observation, but to learn authoritative agent/policy model 45 46
47 48
3 Proposed Method
We firstly explain the notation convention: Ordinary letters, such as x or X, represent scalars or functions with scalar output. Bold letters, such or X, represent vectors or functions with vector output. The i-th element of x is denoted by Ti, while the first k elements of x by 2ick- We use [x]. ||x|| and ||x|| denote the dimension, ln-rrrm and (2-norm of the vector x. An integer set is defined by [n] = .1).
Without loss of generality, estimated state x is assessed by multiple physical models and/or metrics 1. to an h-dimensional error vector, denoted by
e(x exx,)) y) [E, (x. y), Ep, (x. y), .., Ep, y))
(1)
Each concerned ML estimation obtained from an observation y is remediated independently, so y acts constant in the algorithm, which enables simplifying the error notation to c(X) and Er, (x). A better state estimation is sought by minimizing the following accumulated physical error as
min e(xxxx))=)))))) = -Σ h XEX
w. (.,Ep,()), (x),
where the error weights priorly identified by domain experts according to the targeted SAE application. For our problem of interest, the goal is to find a state correction that is within a desired error tolerance, e.g., e(x) where e >> O is a feasibility threshold, determined by domain experts. Thus is not necessary to find a global optimal solution, instead a feasible solution suffices. A typical iterative framework for black-box optimization can be used for this. For instance, at each iteration
r, a set of selected states
(x())
are queried collect their physical errors resulting in set of ==1 A state analysis is then performed according to the error feedback.
ne
state-error pairs {< ex ==1
In the next iteration, new set of states { xx
[((11]] R++1
J==1 are selected to query. This process is repeated until the feasible state x that satisfics e(x*) is found. When designing such a framework, the objective is to find a feasible state x by querying the physical errors as less times as possible because it is time-consuming to collect the errors.
We challenge the difficult setting of choosing only two states to query at each iteration. To ease the explanation, we first present sketch of our proposed GEESE approach in Algorithm []] It starts
from an initial set of randomly selected and queried states {x (xx).)) )
After this, at each
==1
3

---

iteration, only two new states are selected to query based on a novel twin state selection approach that we propose, which selects a potentially near-optimal state for exploitation and a potentially informative state for exploration, resulting in x Cir
(t) and [R CR The selection requires to (t)
perform error analysis for a large set of candidate states, involving both the errors and their gradients. To ease and enable such computation, we develop differentiable surrogate error model é (x, w) to rapidly approximate those error elements that are expensive to evaluate or in need of gradient calculation, and also provide informative gradient guidance with the assistance of error structure. The weights w of the surrogate model are trained using the queried state-error pairs, which start as (xx")e) the algorithm terminates by satisfying the feasibility condition e(x²) S e. Below, we first explain the process of constructing the surrogate model for error approximation, followed by the twin state selection for characterizing the probability distributions of the candidate states and collecting errors, and finally, the implementation of the complete algorithm.
(0) e.), and then expand by including [ eir and X .ek .ee)) each iteration till (1)
N
i=1
Algorithm 1 Sketch of GEESE
1: Randomly select no states and query their physical errors to obtain Do = {xx
no
i=1
2: Train the surrogate error model é (xx w()) with Do
3: for t < T do
4: Select query state x by exploitation and collect the state-error pair
(x (1) (2)
5: Update the feasible state X* X
6: Stop the algorithm if ee(*)) S E
7: Select query state x((*) by exploration and collect the state-error pair (xR OR
(t))) (()
8: Expand the training data D, using the two pairs (x(() , e17 and (x() 0x2
(5)
9: Keep training the surrogate error model using D,. resulting in è (x, (()))
end for
11: output x*
3.1 Hybrid Neural Surrogate Error Models
We start from an informal definition of implicit and explicit emrors. Among the set of h error elements in Eq. ). those that are expensive to collect or to perform gradient calculation are referred to as implicit errors. These can include cases where the system is too complicated and needs much more time to calculate the gradient than that of network backpropagation: or where the system is indifferentiable, such as the physical model of spectroscopy [15) and aeroengine (26) containing database or map. In addition to implicit errors, the remaining are explicit errors. We order these error
elements so that the first l elements {Ep, (x)) implicit while the remaining {Ep, (x))==<++ are explicit. Our strategy is to develop a surrogate for each implicit error element, while directly calculate each explicit error.
Taking advantage of the robustness of ensemble learning 51 52
we propose to estimate the implicit errors by an ensemble of multiple base neural networks. Each base neural network is fully connected with mapping function Φ(x, w) RD x R w/ + R² taking the D-dimensional state space R² as its input space, while returning the approximation of the & implicit errors by its lc output neurons. An example of such a state space is a space of D = 2 dimensions, where the two dimensions correspond to the temperature and concentration states from spectroscopy. Another example is state space of D = 11 dimensions with each state corresponding to design parameter for aeroengine design, for which we provide more details in Section 2 and Appendix A The network weights are stored in the vector w. We train L individual base networks sharing the same architecture, while obtain the final prediction using an average combiner. As result, given a state estimation x, the estimate of the implicit error vector is computed by
1 = T, Σ
éim (x, (wi))
x (x, w:),
(3)

---

and thus, the accumulated physical error is approximated by
k
h
(x, (ww)/-n)
Σ
w T.
のe (&, wi)
+
Σ -k+11
wy Ep (x)
approximated implicit error
true explicit error
We refer to Eq. evaluation.
(4) as a hybrid surrogate error model including both approximated and true error
The weights of the base neural networks (wi) are trained using a set of collected state-error pairs, e.g., D {(&, e,), In our implementation. bootstrapping sampling [53] is adopted to train each base neural network independently, by minimizing a distance loss between the estimated and collected implicit errors, as
min (x,,) [dist (Φ (8, w.). e1:x)]
(5)
A typical example of the distance function is dist(e, e) | ell
Here, we choose to estimate each element of the implicit error vector, rather than to estimate a scalar value of the weighted error sum, because the structural information of the error vector can directly contribute in training through its associated gradient information. When estimating the weighted sum directly, it is in a way to restrict the training loss to a form loosely like (é (w) - |e]l)) )2 which negatively affects the information content of the gradient information. We have observed empirically that, the proposed individual error estimation leads to improvements in training the exploitation generator, compared to using the weighted error sum, see ablation study (1) in Table
3.2 Twin State Selection
selection strategy, i.e., twin state selection (TSS), for querying two individual states at each iteration is proposed, one for exploration and one for exploitation, respectively. The objective of TSS is to substantially reduce the cost associated with physical error collection. turn, this translates to the formidable challenge of designing selection process, which maximizes the informativeness of the associated physical error collection subject to minimizing query times. It is obviously impractical and inaccurate to adopt the naive approach of choosing directly state by searching the whole space. Instead, we target at a two-folded task, researching (1) which candidate set of states to select from and (2) how to select.
By taking advantage of developments in generative AI, we construct generative neural networks to sample the candidate states. Specifically, employ a latent variable z E Rdd which follows a simple distribution, e.g., uniform distribution z ~ U (I-a, a)"), and a neural network G 0) Rd x RI - RD_ The transformed distribution (G(z, 0)) is then used to model the distribution of a candidate set. Thus, the task of candidate selection transformed into determining the neural network weights O for the generator G.
In general, exploitation attempts to select states close to the optimal one, whereas exploration attempts to select more informative states to enhance the error estimation. There are various ways to simulate the exploitation and exploration behaviours. For instance, in conventional black-box optimization, e.g., Bayesian optimization and GA, exploitation and exploration are integrated within single state selection process [54). while in reinforcement learning, balance trade-off approach is pursued [55 39]. Our method treats them as two separate tasks with distinct strategies for constructing generators and selecting states.
ExploITation: To simulate the exploitation behaviour, the exploitation generator Gyr is trained at each iteration by minimizing the expectation of the physical enror estimate, using the hybrid surrogate error model
(é (G (w L
0 G. = arg min E2-U([-aaaf RER
where the base networks from the last iteration are used and we add the subscript / 1 to the weights of the error network for emphasizing. Finally, among the candidates generated by GIT with its trained
5

---

weights GT²
we select the following state
& (1)
XII = arg
min x~pl s 1002
11=1)
If the queried error less than the feasibility threshold, i.c., eir S e, this selected state is considered acceptable and the iteration is terminated. Otherwise, it is used to keep improving the training of the surrogate error model the next iteration.
to query its physical error by Eq.
resulting in the state-error pair
(1) ( T
ExploRation: To simulate the exploration behaviour, state that does not appear optimal but has the potential to complement the surrogate error model should be selected. We use an exploration generator GR to generate candidates. To encourage diversity as to facilitate exploration, we assign generator random weights sampled from a simple distribution, e.g.-
L GR
N (0, (0, (10x)))))
We do not intend to train the exploration generator Gr. because any training loss that encourages exploration and diversity can overly drive the base networks to shift focus in the state space and cause instability in the integrated algorithm. Such an instability phenomenon, caused by training Gg, is demonstrated in the ablation study (2) in Table
we consider the state, for which the base networks are the least confident about to estimate the implicit errors, as more informative. ensemble of base neural networks estimate the error, the standard deviations of the base network predictions serve as natural confidence measures [56), which are stored in k-dimensional vector:
By adopting the idea of active exploration via disagreement 56 57
Since
(x (
L 1=1 ) [ (for (&, w.)kia) ....,a (for (&, (2)
(9)
The state maximizing disagreement, i.e., an accumulated standard deviation, between the base networks, is selected given as
max a (w. (t-1))) & (
i=1 wx,
(10)
where the row vector W [wi, W2, .... wa] stores the implicit error weights. The state-error pair &R ok is obtained after error collection.
Surrogate Model Update: To initialize the algorithm, priorly collect set of state-erro pairs Do {x,, ) for randomly selected states. Next, at each iteration f, two new states are selected and their physical errors are calculated, thus resulting in two new training examples to update the
surrogate error model, and an expanded training set D, = Di-1 U
(t)
e
(4)
(1) In
our implementation, the base neural network weights w (t-1) obtained from the previous iteration
further fine tuned using the two added examples x11 eir and (x()) cx
(t)
(1) (t)
well the N
examples sampled from the previous training set D,-1.
3.3 Remediation System and Implementation
Given MI. estimation &, the remediation system collects its physical error vector in Eq. QT.. then calculates the accumulated error from the objective function of Eq. (2) and compares it to the feasibility threshold >> O. When the error exceeds the threshold, the GEESE algorithm is activated to search a feasible estimation X such that e (X*) < by querying the physical error as few times as possible. Algorithm 2 outlines the pscudocode of GEESE while Fig illustrates its system architecture. Our key implementation practice is summarized below.
project repo: ht https: http:///hhuuu.commaalaaaa/n////a

---

then Algorithm 2 GEESE
Require: Physical
model (and metrics) (P:) and error weights (w;) feasibility
threshold > O, training frequency coefficient 6g. focus coefficient e > 0, maximum iteration
numbers T' and Te, early stopping threshold e, > 0, initial training data size N
Ensure: An acceptable state X with E(x*) S
initial base neural network weights (w." (0) " S== number of early
Initialize: iteration index
stopped base neural networks n, initial exploitation neural network weights 0
2: Sample Zrr for exploitation generator GIT
3: Randomly select N states to query their physical errors to obtain Do exx,,::
4: for & T do
5: Set the added training dataset AD, =
6: Update ((t) by training with Eq. (6) approximated by Zir for up To dGl 2m +
iterations
7: Select exploitation query state x by Eq. S ce then
approximated by x
8: if
{ i=
Collect the new state-error pair AD, AD, (x:(() c(00 set x- xx(1)
9:
10: end if
11:
if
El= w; WrEr, (x*) S
12:
Stop the algorithm
13:
end if
14:
Sample ((0) by Eq.
15: Select exploration query state XR by Eq. (10) approximated by xx
16:
Collect the new state-error pair AD, AD; U &R eR
(t)
update the training data D,
D U AD;
17:
Obtain D, by sampling randomly N state-error pairs from Dz-1 for each base neural network.
Prepare training datasets {Di) where D; = AD, U D;
18: Update { (w] :-1
by training each base neural network using D; by Eq. (5) for up to Te
iterations, and count the number of early stopped base neural networks ne
19: end for
Empirical Estimation: Eqs. 6 and (10) require operations performed over probability distribu- tions. In practice, we approximate these by Monte Carlo sampling. For Eq. (6). we minimize instead the average over the sampled latent variables Zir { with z; ~ U ([-ar amr"") and this is fixed in all iterations. The search space of Eq. (7) is approximated by state set computed from Nir
Nir
ZI using the trained generator, i.e., x = =( GiT 2,, (.))
Similarly, the search space of Eq.
i=1" N2
approximated by a state sample x(()) = GR (Z2, GG
} where z; U (I-ag, og]").
110
Early Stopping: When training the base neural networks for implicit enror estimation. in addition to the maximum iteration number Te. early stopping of the training is enforced when the training loss in Eq. (5) is smaller than preidentified threshold c,. As result, higher number n. of early stopped base neural networks indicates a potentially more accurate error estimation. This strengthens the confidence in training the generator GIT by Eq. (6 that the trained base neural network from the previous iteration. In other words, when the base neural network are not sufficiently well trained, is not recommended to put much effort in training the generator, which relies on the estimation quality. Therefore, we set the maximum iteration number Ta for training GIT in proportional to ne. i..., To dG 2n 1J. where åG is training frequency coefficient.
Failed Exploitation Exclusion: The state selection motivated by exploitation aims at choosing an (1) XIr with comparatively low physical error. To encourage this, focus coefficient c introduced,
7

---

Ep
Impiltele Eiver Explier Error
Explicit Ertor
Latent Vector Zyz
Physteal Error
a 3 ... L0 oo
8t
Syri
est
Latent Vector Zx
A 8% 6n
o'
8x
Accept
Networks
Hybrid Surrogate Errur Modly/
Back propagation
Forward computation
Figure The workflow of GEESE; If the estimation from an MI. estimator fails the physical evaluation Ep, GEESE is activated. The error estimated by hybrid surrogate error model is used to train the exploitation generator GIT. Two candidate state sets are generated by GIT and exploration generator Gr. and finally, two states X ÅIT and åR are selected by the surrogate error model and fed into Ep for evaluation and data collection. The process terminated till e( e(**) S.
which, together with the feasibility enror threshold >> 0, is used to exclude potentially failed state with a high estimated error, i... è (x, {wi)==)) > cc, to avoid an unnecessary query.
4 Experiments and Results
We test the proposed approach GEESE on three real-world engineering inverse problems, including aero engine design [42. electro-mechanical actuator design (58) and pulse-width modulation of
engine to satisfy the thrust and fuel consumption requirement (observation). The second problem is to find 20 design parameters (state) of mecchr--mecaaical actuator to satisfy requirements for overall cost and safety factor (obversation). And the third problem is to find a group of 30 control parameters (state) of a 13-level inverter satisfy the requirements for distortion factor and nonlinear factor (observation). Details of these problems along with their physical models and metrics for evaluation are explained in supplementary material (Section A). We compare with a set of classical and slate-of-thc-as -bx optimization techniques, including Bayesian Optimization with Gaussian Process (BOGP), GA [2T]. PSO 22 CMAES (60, ISRES [I], NSGA2 (22) and UNSGA3 [6), as well as the recently proposed work SVPEN [4), which employs RL in solving SAE inverse problems. These techniques are chosen because they are effective at seeking solutions with the assist of actual physical evaluations.
13-level inverters [59]. The first problem is to find eleven design parameters (state) of
In practice, different types of simulators exist for the same problem. For instance, for problem a simulator mentioned in [64] with high fidelity takes 10.3 seconds x 309 iterations = 53 minutes to obtain a converged simulation, while another simulator in [65] with a much lower fidelity can realize real-time simulation. Since we aim at a research scenario that attempts to search feasible state without querying too much data and without setting a high standard on data quality, we choose to use faster simulators with lower but reasonable fidelity in our experiments. For the three studied problems, each simulator takes no more than five seconds to run. Since the computing time varies when changing simulators, we report the performance by query times instead of actual computing time. Accordingly, we adopt two metrics to compare performance. First, we set maximum budget of T = 1, 000 query times for all studied problems and compared methods, and test each method on each problem individually with 100 experimental cases, each case corresponds to a concerned ML state estimation. The setup of the experimental cases is described in Appendix [A] of supplementary material. we measure the number of experiments out of 100 where a method fails to correct the concerned estimation when reaching the maximum query budget, and refer to it as the failure times Nialure- Also, the average number of queries that a method requires before finding a feasible state in an experiment, is reported over 100 experiments, and referred to as average query times N Nquery. A more competitive algorithm expects smaller Niaiure and Nguery.
We report the adopted hyper-paramend and model setting for GEESE: The common hyperparameter settings shared between all three studied problems include T, 40, e, = 1 4 and N 64, and the learning rates of 1 and -1, for training the exploitation generator and base neural networks, respectively. Different focus coefficients of = 1.5, and (set in increasing fashion) are used for problems 1, and 3, respectively, due increased problem complexity in relation to their
8

---

Table 1: Performance comparison of the compared methods, where the best is shown in bold, while the second best is underlined
Problem 1
Problem
Problem
Algorithm
State Dimension:11
State Dimension:20
State Dimension:30
Failure times
Query times
Failure times
Query times
Failure times
Query times
BOGP GA
0
#1.51
97
973.76
112.66 =229 98
0
64.00 =0.00
0
130.56 #63.31
13
231,76 #339.71
PSO
0
64.00 =0.00
0
64.00 +0 +0.00
12
244, 16+34,777
CMAES
0
55.67 #3.28
119.44 =41.80
227.42 #312.17
ISRES
0
65, 65.00+0.00
177.64 =80,51
250.05 #350.16
NSGA2
0
64.00 =0.00
139.52 #68.56
232.40 335.94
UNSGA3
0
64.00 =0.00
140.80 80 =79,94 94
227.52 +330.07
SVPEN
100
1000.00#0,00
100
1000.00=0.00
100
1000.00
GEESE (Ours)
0
3,18 +1.98
0
51.65 #33.01
43.56 #65,28
increasing dimensions of the state space. Similarly, an increasing training frequency coefficient dG = 1, and is used for problems I, 2 and 3, respectively, because the problem requires more training iterations as it involves more complex patterns from higher-dimensional state space. The ensemble surrogate model for estimating the implicit errors is constructed as an average layer perceptrons (MLPs) each with three hidden layers consisting of 1024, 2028 and 1024 hidden neurons. The exploration generator Gg is constructed as a single layer perceptron (SLP) and its one-dimensional input is sampled from U (1-5, 5]). For problems and 2 that relatively less complex from an engineering point of view, we directly set the latent space 2 as the state space X' without using any neural network transform in between. Then, directly sample initial state set X The exploitation state is directly optimized iteratively, e.g., by a gradient descent approach based ( to obtain state set X shown in Eq. function value is selected as the exploitation state, i.e.,
multi-
(1 The one with the smallest objective
( X
arg min è (x. (x RE
(11)
Problem involves a special state pattern, requiring an increasing state value over the dimension, i.e., X; Xi+1 < O. To enable the latent variables to capture this, we construct the exploitation generator Gir as an MIP with three hidden layers consisting of 256, 512 and 256 hidden neurons. Also, to avoid generation collapse [66] in problem 3, regularization term has been added to the training loss in Eq. (6), resulting in the following revised training to encourage state diversity, as
0 arg min (1_ è (Gn(2,,), 0), {w! RE
(w (0] " max (0.0288 σi (z. 0).0 (12)
where 01 (z 0) denotes the standard deviation of the first state element generated by Gir. We
encourage it to shift away from the collapsed point but not overly spread, by bounding σι with a
portion of the standard deviation of a uniform distribution, e.g., 0.288, and the portion 0.288 0.0288 observed empirically effective. The spread control only needed for the first state as the remaining states follow by X, X++1 < O. Configurations of the competing methods, together with extra information on GEESE, are provided in Appendix of supplementary material.
10
4.1 Results and Comparative Analysis
Table summarizes the results of the compared methods for the three problems, obtained with a feasibility threshold of e = 0.075, which reflects high challenge with low eиrоr tolerance. It can be observed that GEESE has the least failure times Naalure on all three problems. In problem 3, especially, GEESE succeeds with no failure while most other methods have more than 10 failures. This is highly desired characteristic for a remediation system with low error tolerance. In addition, GEESE also has the least query times Nouery in all three problems, indicating the best efficiency. We report additional results in Appendix [ of supplementary material by varying the feasibility threshold e and the initial sample size N. where GEESE also achieves satisfactory performance in general, while outperforming other methods in handling higher-dimensional problems with lower error tolerance. SVPEN 42 cannot return a feasible correction in 1000 queries in all experiments, as its core supporting RL requires a lot more queries than other optimization based techniques.

---

4.2 Ablation Studies and Sensitivity Analysis
To examine the effectiveness of the key design elements of GEESEE, we perform a set of ablation studies and report the results in Table 2 using problem 1 with small feasibility threshold 0.05 indicating low error tolerance. The studies include the following altered designs: (1) Estimate directly the implicit error sum using an MLP with the same hidden layers but one single output neuron. (2) Train the exploration generator GR by using an approach suggested by [57]. (3) Remove the early stopping design. (4) Remove the focus coefficient.
Results show that estimating the implicit error sum worsens the performance. As explained earlier in Section 3.1] this is because the structural information in gradient is lost in error sum estimation, causing ambiguous update when training GIT. and consequently requires GEESE to make more error queries. Also training GR worsens the performance compared to just assigning random network weights to GR without training. As previously explained in Section 3.2 this is because training GR can frequently shift the focus of the surrogate eπror model and, thus, impact on the stability of the optimization process. Both early stopping and focus coefficient play an important role in GEESE, where the former prevents GEESE from overfitting and the latter helps avoid unnecessary queries. Additional results on hyperparameter sensitivity analysis for GEESE are provided in Appendix [D] of supplementary material. The results show that GEESE is not very sensitive to hyperparameter changes and allows a wide range of values with satisfactory performance, which makes GEESE easy to be tuned and used in practice.
Table 2: Results of ablation studies reported on problem 1, where a better performance is highlighted in bold.
(1): Individual vs Error Estimation
(2): Effect of Exploration Training
Surrigate Error Model
Query times
Standard deviation
Exploration style
Query times
Standard deviation
Estimute elements
20.20
16.37
w/o training
32.64
22.82
Estimate error sum
23.26
21.18
with training
41.32
97.15
(3): Effect of Early stopping
(4): Effect of Focus Coefficiett
Schedule
Query times
Standard deviation
Schedule
Query times
Standard deviation
with carlystop
20.20
16.37
with focus coefficient
20.20
16.37
w/o earlystop
32.80
17.84
w/o focus coefficient
27.19
19.36
5 Discussion and Conclusion
We have proposed a novel physics-driven optimization algorithm GEESE to correct ML estimation failures in SAE inverse problems. To query less frequently expensive physical evaluations, GEESE uses a cheaper hybrid surrogate error model, mixing an ensemble of base neural networks for implicit error approximation and analytical expressions of exact explicit errors. To effectively model the probability distribution of candidate states, two generative neural networks are constructed to simulate the exploration and exploitation behaviours. In each iteration, the exploitation generator is trained to find the most promising state with the smallest error, while the exploration generator is randomly sampled find the most informative state to improve the surrogate error model. These two types of selection are separately guided by the approximated error by the ensemble and the disagreement between its base neural networks. The element-wise error approximation promotes more effective interaction between the surrogate error model and the two generators. Being tested on three real- wolld engincering inverse problems, GEESE outperforms all the compared methods, finding a feasible state with the least query number with no failure under the low error tolerance setup.
However, there are still challenges to address in the future, particularly for very high-dimensional inverse problems. Such problems are in need of larger and more complex model architecture to accommodate their more complex underlying patterns, and thus impose challenge on training time and data requirement. Computation expense should not only consider the query cost of physical evaluations but also the learning cost of such models. Flexible neural network architectures that allow for embedding domain specific or induced knowledge in addition to simulation data and their training, as well as their interaction with the main solution model, e.g., an MI. estimator for inverse problems, are interesting directions to pursue.
10

---

References
[1] Christopher S. Goldenstein, R. Mitchell Spearrin, Jay B. Jeffries, and Ronald K. Hanson. Infrared laser-absorption sensing for combustion gases. Progress in Energy and Combustion Science, 60:132-176, 2017.
[2] Jack D. Mattingly, William H. Heiser, David T. Pratt, Keith M. Boyer, and Brenda A. Haven. Aircraft Engine Design, Third Edition. American Institute of Aeronautics and Astronautics, Inc., Washington, DC, 2018.
[3] Christopher S. Goldenstein, Ian A. Schultz. Jay B. Jeffries, and Ronald K. Hanson. Two-color absorption spectroscopy strategy for measuring the column density and path average temperature of the absorbing species in nonuniform gases. Applied Optics, 52(33):7950-799) 2013.
[4] T.-C. Jim Yeh, S. Liu, R. J. Glass, K. Baker, R. Brainard, D. Alumbaugh, and D. LaBrecque. A geostatistically based inverse model for electrical resistivity surveys and its applications to vadose zone hydrology. Water Resources Research, 38(12))14-1-14-- December 2002.
[5] Keith T. Butler, Daniel W. Davies, Hugh Cartwright, Olexandr Isayev, and Aron Walsh. Machine learning for molecular and materials science. Nature, 559(7715):547-555)) July 2018.
[6] Lianlin Li, Hengxin Ruan, Che Liu, Ying Li, Ya Shuang, Andrea Alü, Cheng- Wei Qiu, and Tie Jun Cui. Machine-learning reprogrammable metasurface imager. Nature Communications, 10(1):1082, December 2019.
[7] Zijiang Yang, Dipendra Jha, Arindam Paul, Wei-keng Liao, Alok Choudhary, and Ankit Agrawal. A General Framework Combining Generative Adversarial Networks and Mixture Density Networks for Inverse Modeling in Microstructural Materials Design. 2021.
[8] Shengduo Chen, Yaowei Sun, Dachuan Li, Qiang Wang. Qi Hao, and Joseph Sifakis. Runtime Safety Assurance for Learning-enable Control of Autonomous Driving Vehicles. In 2022 International Conference on Robotics and Automation (ICRA), pages 8978-899 8984, Philadelphia,
PA, USA, May 2022. IEEE.
[9] Ilir Murturi and Schahram Dustdar. DECENT: Decentralized Configurator for Controlling Elasticity in Dynamic Edge Networks. ACM Transactions on Internet Technology, 22(3):1-21,
August 2022.
[10] Kei Terayama, Masato Sumita, Ryo Tamura, and Koji Tsuda. Black-box optimization for automated discovery. Accounts of Chemical Research,
2021.
[11] Fernando Nogueira. Bayesian Optimization: Open source constrained global optimization tool for Python, 2014.
[12] John Peurifoy, Yichen Shen, Li Jing, Yi Yang, Fidel Cano-Renteria Brendan G DeLacy, John D Joannopoulos, Max Tegmark, and Marin Solja. Nanophotonic particle simulation and inverse design using artificial neural networks. SCIENCE ADVANCES, page 8, 2018.
[13] Bin Xia and Da-Wen Sun. Applications of computational fluid dynamics (cfd) in the food industry: review. Computers and electronics in agriculture, 34(1-3):5-24, 2002.
[14] Iulia M Georgescu, Sahel Ashhab, and Franco Nori. Quantum simulation. Reviews of Modern Physics, 86(1):153, 2014.
[15] Roman V. Kochanov, I. E. Gordon, L. S. Rothman, P. Wcisto, C. Hill, and J. S. Wilzewski. HITRAN Application Programming Interface (HAPI): comprehensive approach working with spectroscopic data. Journal of Quantitative Spectroscopy and Radiative Transfer, 177:15- 30, 2016.
[16] Gennady V Alekseev and Dmitry A Tereshko. Particle swarm optimization-base algorithms for solving inverse problems of designing thermal cloaking and shielding devices. International Journal of Heat and Mass Transfer, 135:1269-1277, 2019.

---

[17] P Honarmandi, V Attari, and R Arroyave. Accelerated materials design using batch bayesian optimization: A case study for solving the inverse problem from materials microstructure to process specification. Computational Materials Science, 210:1 11 1417, 2022.
[18] Fei Shen, Fan Fan, Rui Wang, Yue Wang. Songyan Yang. Qiong Wu, Pascal Laugier, Xiran Cai, and Haijun Niu. Inverse problem in resonant ultrasound spectroscopy with sampling and optimization: A comparative study on human cortical bone. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 69(2):650-661, 2021.
[19] Michael Hinze, René Pinnau, Michael Ulbrich, and Stefan Ulbrich. Optimization with PDE constraints, volume 23. Springer Science & Business Media, 2008
[20] Christopher Williams and Carl Edward Rasmussen. Gaussian Processes for Machine Learn- ing, volume 2. MIT press Cambridge, MA, 2006.
[21] Seyedali Mirjalili. Genetic Algorithm, volume 780, pages 43-55. Springer International Publishing, Cham, 2019.
[22] James Kennedy and Russell Eberhart. Particle swarm optimization. In Proceedings of ICNN'95- international conference on neural networks, volume 4, pages 1942-1948. IEEE, 1995.
[23] Jorge Nocedal and Stephen J Wright. Numerical optimization. Springer, 1999.
[24] Brandon Amos and J. Zico Kolter. Optnet:
Differentiable optimization as layer in neural networks. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML' 17, page 136-145 JMLR.org, 2017.
[25] II Yong Chun, Zhengyu Huang, Hongki Lim, and Jeffrey A. Fessler. Momentum-Net: Fast and Convergent Iterative Neural Network for Inverse Problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4915-4931, 2023.
[26] Joachim Kurzke and Ian Halliwell. Propulsion and Power: An Exploration of Gas Turbine Performance Modeling. Springer, 2018
[27] David Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In International conference on machine learning, pages 773-782. PMLR, 2019.
[28] Jie Chen and Yongming Liu. Neural Optimization Machine: A Neural Network Approach for Optimization, August 2022.
[29] Dong Han, Wenli Du, Xinjie Wang, and Wei Du. A surrogate-assisted evolutionary algorithm for expensive many-objectio optimization in the refining process. Swarm and Evolutionary Computation, 69:100988, 2022.
[30] SeyedMahmood VaeziNejad, SeyedMorteza Marandi, and Eysa Salajegheh. A Hybrid of Artificial Neural Networks and Particle Swarm Optimization Algorithm for Inverse Modeling of Leakage in Earth Dams. Civil Engineering Jourmal, 509):2041-2057, 2019.
[31] Dianjing Liu, Yixuan Tan, Erfan Khoram, and Zongfu Yu. Training Deep Neural Networks for the Inverse Design of Nanophotonic Structures. ACS Photonics, 3665:196, April 2018.
[32] Gabriel Villarrubia, Juan F. De Paz, Pablo Chamoso, and Fernando De la Prieta. Artificial neural networks used in optimization problems. Neurocomputing. 272:10-16, January 2018.
[33] Aviral Kumar and Sergey Levine. Model Inversion Networks for Model-Basee Optimization, December 2019.
[34] Sihyun Yu. Sungsoo Ahn, Le Song, and Jinwoo Shin. Roma: Robust model adaptation for offline model-based optimization. Advances in Neural Information Processing Systems, 2021.
[35] Justin Fu and Sergey Levine. Offline Model-Based Optimization via Normalized Maximum Likelihood Estimation, February 2021.
12

---

[36] Brandon Trabucco, Aviral Kumar. Xinyang Geng, and Sergey Levine. Conservative objective models for effective offline model-based optimization. In International Conference on Machine Learning, pages 10358-10368. PMIR, 2021.
[37] Sen Zhao, Erez Louidor, and Maya Gupta. Global optimization networks. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, edi- tors, Proceedings of the 39th International Conference on Machine Learning. volume 162 of Proceedings of Machine Learning Research, pages 26927-26957. PMLR, 17-23 Jul 2022.
[38] Fengji Yi, Wenlong Fu, and Huan Liang. Model-based reinforcement learning: A survey. 2018.
[39]
Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, and Sergey Levine. Soft Actor-Critic Algorithms and Applications. arXiv, 2018.
[40] S Phaniteja, Parijat Dewangan, Pooja Guhan, Abhishek Sarkar, and K Madhava Krishna. A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots. In 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), pages 1823. IEEE, 2017.
[41] Bhuvanesh Sridharan, Manan Goel, and Deva Priyakumar. Modern machine learning for tack- ling inverse problems in chemistry: molecular design to realization. Chemical Communications, 58(35):5316-533) 2022.
[42] Ruiyuan Kang, Dimitrios C. Kyritsis, and Panos Liatsis. Self- Physics-Embeedding Network: A General Framework for Inverse Modelling, October 2022.
[43] Daphna Joel, Yael Niv, and Eytan Ruppin. Actor-critic models of the basal ganglia: New anatomical and computational perspectives. Neural networks, 15(4-6)553-544 2002.
[44] Jürgen Schmidhuber. On learning to think: Algorithmic information theory for novel combina- tions of reinforcement learning controllers and recurrent neural world models. arXiv preprint 1rXiv:15/1.09249 2015.
[45] Lorenzo Canese, Gian Carlo Cardarilli, Luca Di Nunzio, Rocco Fazzolari, Daniele Giardino, Marco Re, and Sergio Spanò. Multi-agent reinforcement learning: A review of challenges and applications. Applied Sciences, 11(11))4948, 2021.
[46] Yongshuai Liu, Avishai Halev, and Xin Liu. Policy learning with constraints in model-frea reinforcement learning: A survey. In The 30th International Joint Conference on Artificial Intelligence (IJCAI), 2021.
[47] Kazuo Yonekura and Hitoshi Hattori. Framework for design optimization using deep reinforce- ment learning. Structural and Multidisciplinary Optimization, 10:1709-17 2019.
[48] Xinyu Hui, Hui Wang, Wenqiang Li, Junqiang Bai, Fei Qin, and Guoqiang He. Multi-object aero- dynamic design optimization using deep reinforcement learning. AIP Advances, 1188001853 2021.
[49] Zhenpeng Zhou, Steven Kearnes, Li Li, Richard N Zare, and Patrick Riley. Optimization of molecules via deep reinforcement learning. Scientific reports, 9(1):1-10, 2019.
[50] Jonathan Viquerat, Jean Rabault, Alexander Kuhnle, Hassan Ghraieb, Aurélien Larcher, and Elie Hachem. Direct shape optimization through deep reinforcement learning. Journal of Computational Physics, 428:1 10080, 2021.
[51] Ling Liu, Wenqi Wei, Ka-Ho Chow, Margaret Loper, Emre Gursoy. Stacey Truex, and Yanzhao Wu. Deep neural network ensembles against deception: Ensemble diversity, accuracy and robustness. In 2019 IEEE 16th international conference on mobile ad hoc and sensor systems (MASS), pages 274-282. IEEE, 2019.
[52] Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavanzadeh, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U Rajendra Acharya, et al. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information Fusion, 76:243-297 2021.
13

---

[53] Christopher Z Mooney, Christopher F Mooney, Christopher L Mooney, Robert D Duval, and Robert Duvall. Bootstrapping: A Nonparametric Approach Statistical Inference. Number 95. sage, 1993.
[54] Daniel J Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband, Zheng Wen, et al. A tutorial on thompson sampling. Foundations and Trends@ in Machine Learning, 11(1):1-96, 2018.
[55] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing Atari with Deep Reinforcement Learning, 2013.
[56] Deepak Pathak, Dhiraj Gandhi, and Abhinav Gupta. Self-Supervised Exploration via Disagree- ment, June 2019.
[57] Pranay Shyam, Wojeiech Jaškowski, and Faustino Gomez. Model-Based Active Exploration, June 2019.
[58] Cyril Picard and Jurg Schiffmann. Realistic Constrained Multiobjective Optimization Bench- mark Problems From Design. IEEE Transactions on Evolutionary Computation, 25(2):234-25 246, April 2021.
[59] Abhishek Kumar, Guohua Wu, Mostafa Z. Ali, Qizhang Luo, Rammohan Mallipeddi, Ponnuthu- rai Nagaratnam Suganthan, and Swagatam Das. Benchmark-Suit of real- World constrained multi-objecctvve optimization problems and some baseline results. Swarm and Evolutionary Computation, 67:100961, December 2021.
[60] Nikolaus Hansen. The CMA Evolution Strategy: A Comparing Review, In Jose A. Lozano, Pedro Larrañaga, Iñaki Inza, and Endika Bengoetxea, editors, Towards a New Evolutionary Computation, volume 192, pages 75-102. Springer Berlin Heidelberg. Berlin, Heidelberg. 2006.
[61] T.P. Runarsson and X. Yao. Search Biases in Constrained Evolutionary Optimization. IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 35(2)2233- 243, May 2005.
[62] K. Deb, A. Pratap. S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-IL. IEEE Transactions on Evolutionary Computation, 6(2):182-197, April 2002.
[63] Haitham Seada and Kalyanmoy Deb. A Unified Evolutionary Optimization Procedure for Single, Multiple, and Many Objectives. IEEE Transactions on Evolutionary Computation, 20(3):358-369 June 2016.
[64] Xiaobo Zhang, Zhanxue Wang, and Li Zhou. An integrated modeling approach for variable cycle engine performance analysis. Journal of the Franklin Institute, 360(6):554--5563, May 2023.
[65] Henrique Gazzetta Junior, Cleverson Bringhenti, João Roberto Barbosa, and Jesuino Takachi Tomita Real-Time Gas Turbine Model for Performance Simulations. Journal of Aerospace Technology and Management, 9()):366333 356, August 2017.
[66] Jiachen Zhu, Rafael M. Moraes, Serkan Karakulak, Vlad Sobol, Alfredo Canziani, and Yann LeCun. TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, June 2022.
[67] Hakan Aydin, Onder Turan, T. Hikmet Karakoc, and Adnan Midilli. Exergetic Sustainability Indicators as a Tool in Commercial Aircraft: A Case Study for a Turbofan Engine. International Journal of Green Energy, 12(1):28 40, 2015.
[68] Johan Åkerberg, Mikael Gidlund, and Mats Björkman. Future research challenges in wireless sensor and actuator networks targeting industrial automation. In 2011 9th IEEE International Conference on Industrial Informatics, pages 410-415. IEEE, 2011.
[69] Patrick R Buckley, Gareth H McKinley, Thomas Wilson, Ward Small, William J Benett, Jane Bearinger, Michael W McElfresh, and Duncan J Maitland. Inductively heated shape memory polymer for the magnetic actuation of medical devices. IEFE transactions on biomedical engineering, 53(10)200552208 2006.
14

---

[70] Xidong Tang, Gang Tao, and Suresh M Joshi. Adaptive actuator failure compensation for nonlinear mimo systems with an aircraft control application. Automatica, 43((1):1869-1888 2007.
[71] Amarendra Edpuganti and Akshay Kumar Rathore. Optimal Pulsewidth Modulation for Common-Moon Voltage Elimination Scheme of Medium- Voltage Modular Multilevel Converter- Fed Open-End Stator Winding Induction Motor Drives. IEEE Transactions Industrial Electronics, 64(1):848 856, January 2017.
[72] G Vasuki, M Vinothini, SB Rushmithaa, K Karthik Kumar, AS Kamaraja, and M Willjuice Iruthayarajan. 13-level inverter configuration with reduced auxiliary circuit for renewable energy applications. 2021 5th International Conference on Electronics, Communication and Aerospace Technology (ICECA), pages 101-105. IEEE, 2021.
[73] Ashish Kumar, Ram Kumar, Abhishek Kumar, and PR Thakura. Design and development of 13-level multilevel inverter for hybrid electric vehicles. In 2017 International Conference on Intelligent Computing and Control Systems (ICICCS), pages 121-126. IEEE, 2017.
[74] Yuanmao Ye, Shikai Chen, Xiaolin Wang, and Ka-Wa Eric Cheng. Self-balanced 13-level in- verter based on switched capacitor and hybrid pwm algorithm. IEEE Transactions Industrial Electronics, (8(6))4877-48333 2020.
[75] Julian Blank and Kalyanmoy Deb. Pymoo: Multi-objective optimization in python. IEEE Access, 8:89497-89500 2020.
15

---

A Studied Inverse Problems
A.1 Problem Description
Problem 1, Turbofan Design: Turbofan is of the most complex gas acro engine systems, and is the dominant propulsion system favoured by commercial airliners [26]. The inverse problem for turbofan design is to find a group of design parameters modelled as the state to achieve the desired performance modelled as observation. The observation includes two performance parameters, including the thrust Y/ and the thrust specific fuel consumption yy- The state includes 11 design parameters that control the performance of the engine, including the bypass ratio Thp- the fan pressure ratio Man- the fan efficiency !/inn the low-pressure compressor pressure ratio SLC. the low-pressure compressor efficiency Th.c, the high-pressure compressor pressure ratio THC, the high-pressure compressor efficiency тHCC the combustor efficiency VJcc, the combustion temperature in the burner TB. the efficiency of high-pressure turbine 7/тТ and the efficiency of low-pre www-presuuee turbine 'Lr- Following the same setting as in [2]], the goal is to estimate the design parameters that can achieve the performance of a CFM-56 turbofan engine, for which the thrust should be 121 KN and the thrust specific fuel consumption should be 10.63 g/(kN.s) 677. This corresponds to the observation vector y [yt, y/ = [121, 10.63). The 100 experiment cases tested on this problem differ from the state to correct, which randomly sampled from the feasible region of the design parameter space provided by [42]. Table reports the allowed range of each design parameter, which all together define the feasible region.
Table 3: Feasible region of the design parameter space for problem 1.
Range
Tbp
Ttan
TLC
THC
TB
Taan
1HC
TLC
T]8
THT
7LT
Min
5
1.3
1.2
8
1300 K
0.85
0.82
0.84
0.95
0.86
0.87
Max
6
2.5
2
15
1800 K
0.95
0.92
0.94
0.995
0.96
0.97
Problem 2, Electro-mechanical Actuator Design: An electro-mechanica. actuator is a device that converts electrical energy into mechanical energy [68], by using combination an electric motor and mechanical components to convert an electrical signal into mechanical movement. It commonly used industrial automation medical devices[69], and aircraft control systems (70), etc. We consider the design of an electro-mechanical actuator with a three-stage spur gears. Its corresponding inverse problem is to find 20 design parameters modelled as the state, according to the requirements for the overall cost ye and safety factor ys modelled the observation. The 100 experiment cases tested on this problem differ from the observation y = [ye. ya). We have randomly selected 100 combinations of the safety factor and overall cost from the known Pareto front [58], which is shown in Fig. 2a For each observation, the state to correct is obtained by using an untrained MI. model to provide naturally failed design.
180
160
140
Distortion factor
120
100
2-
80
60
40
20
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0
0.5
1.0
1.5
2.0
Safety factor
Nonlinear factor
(a)
Figure 2: Illustration of the 100 used test cases in the 2-dimensional observation space for problem 2 (subfigure a) and (subfigure b).
Problem 3, Pulse-width Modulation of 13-level Inverters: Pulse-width modulation (PWM) of n-level inverters is technique for controlling the output voltage of an inverter that converts DC power into AC power [71]. It modulates the duty cycle of the output waveform of the inverter,
16

---

thereby affecting the effective value of the voltage applied to a load. Particularly, an PWM of 13-level inverter adjusts its output waveform using 13 power switching devices to achieve higher precision, which is widely used in renewable power generation systems 722,, electric vehicles [73] and industrial automation equipment [74]. results in typical inverse problem of finding the suitable control parameters including 30 switch angles modelled as the state, according to the requirements of the distortion factor Ya (which measures the harmonic distortion of the output waveform) and the nonlinear factor V/. (which avoids the malfunctioning of the inverter and the connected load) modelled as the observation. As problem 2, the 100 experiment cases tested on this problem also differ from the observation, i.e. y [yd Vnl- They correspond to 100 randomly selected combinations of the distortion and nonlinear factors from the known Pareto front in [59], which are shown in Fig. 2b For each observation, the state to correct is obtained by using untrained ML model to provide naturally failed estimation.
A.2 Physical Evaluation
We describe in this section how the physical evaluations are conducted, more specifically, how the physical errors are assessed. Overall, it includes the observation reconstruction error, which is based on the difference between the given observation and the reconstructed observation from the estimated state. For different problems, different physical models used to simulate the reconstruction. It also includes the feasible domain error, which examines whether the estimated state is within a feasible region of the state space, and this region is often known for a given engineering problem. Apart from these, there are also other problem-specific errors.
A.2.1 Problem 1
Observation Reconstruction Error:
The gas turbine forward model [42 is used to simulate the performance of the turbofan engine. It is constructed through the aerodynamic and thermodynamic modelling of the components in turbofan engine, where the modelled components include the inlet, fan, now-pressure and high-pressure compressors, combustor, high-pressure and low-pressure turbines, core and fan nozzles, well as through considering the energy losses. This model transform the input of state into physically reasonable output of observation, which is the thrust y: and fuel flow yf of the engine. Let F(x) denote forward model. In problem the performance requirement is [121, 10.63). thus, for a estimated state &, the reconstruction error is
specifically = [yt, Myl
[| yilli
e, (x))=
Σ
(13)
2y,
where, when respectively equals to 1 or 2, (X)) and (2( are the estimated thrust and fuel con- sumption in the engine case, respectively. Because the magnitude of the thrust and fuel consumption are different, we use the relative error to measure the reconstruction error of the two observation elements.
Feasible Domain Error: In aero engine design, the design parameters cannot exceed their feasible region and such a region has already been identified by existing work 42 as in Table 3 For the i-th dimension of an estimated state क; (an estimated design parameter), and given its maximum and minimum allowed values F mmax and Tmin. we define its feasible domain error by
( max x Fmin - 1,0 + max (-
Tmin 0
(14)
Tmax Xmin
Fmaa
Zmia
After normalization, all the feasible values within the range of j0, 1]. while the non-feasible
ones outside. The above error simply examines how much the normalized state value exceeds or below O. We compute accumulated feasible error for all the 11 design parameters, given by (x) = e!
Design Balance Error: Another desired property by aero engine design is low disparity among the values of the design parameters after normalizing them by their feasible ranges, which indicates a more balanced design, offering better cooperation between the design components and resulting in lower cost [42 26]. Standard deviation suitable measure to assess this balance property, resulting in another physical error
eo ex(x) = (f
x; ( Fmin
11
(15)
Immax - Fmii t=1
17

---

where (.) denotes the standard deviation of the elements from its input set.
Accumulated Physical Error: The above three types of errors combined to form the following accumulated physical error:
8(x) = ex + (x) + O. (X).
(16)
The weights are given as 1, 0.1 and 0.1, respectively. This is because the reconstruction error determines whether the estimated state is feasible, while the other two errors are used to further improve the quality of the estimated state from the perspective of the design preference. Here e, (x) is obtained using forward simulation process thus is implicit error, while ey and ea have analytical expressions and simple gradient forms, and thus are explicit errors.
A.2.2 Problem 2
Observation Reconstruction Error: The used forward model for electro-mechanical actuator design a performance simulation model, considering a stepper motor, three stages of spur gears and a housing to hold the components (i.e., stepper motor, and three stages of spur gears) [58]. It consists of a physical model that predicts its output speed and torque and component-speciffe constraints, a cost model and geometric model that creates 3-D meshes for the components and the assembled system. The integrated model predicts the observation = {ye, ). and is named as the "CS1" model in [88) After reconstructing by CS1 the safety factor ys and total cost ye. from the estimated design parameters X, the reconstruction error is computed using Eq. 13
Feasible Domain Error:
The same feasible domain error ey as in Eq. is used for each design parameter of problem 2. The only difference is that the allowed parameter ranges for defining the feasible region have changed. We use the region identified by [58]. There are 20 design parameters, thus ey is an average of 20 individual errors.
Inequality Constraint Error:
We adopt another seven inequality constraints provided by the forward model [58 examine how reasonable the estimated design parameters are. These constraints do not have analytical forms, and we express them as e, (X) < 0 for i = 1,2,.... 7. Based on these, we define the following inequality constraint error
Cel xx)
max (xx))c()))0))
(17)
1=1
Accumulated Physical Error: We then combine the above three types of errors, given as
é(x) = (x) + 0.1ey(x) + ex(x),
(18)
where both (x) and (x) are implicit errors computed using black-box simulation model, while ey is an explicit error. In this case, increase the weight for inequality constraint error to be the same as the reconstruction error, this is because we regard the implicit errors irrespective of their
types as the same. Of course, one can also use different weights for different types of errors according to their expertise.
A.2.3 Problem 3
We use the forward model from 59 to reconstruct the observation for the 13-level inverter. It takes the control parameters as the input and returns the distortion factor ya and the nonlinear factor yn- Based the reconstructed ya and yn. the observation reconstruction error e, is computed by Eq. [13
the same way as in problems and 2. Similarly, the same feasible boundary error é as in Eq. (4
is computed, but the feasible region is different where the range of [0, 51 is applied for all the 30
control parameters, which defined in (59]. A similar inequality constraint error as in Eq. [7 is
used, which contains 29 inequality constraints in the form of
(x) X4 x < O, for i 1,2,... 29.
(19)
Finally, the accumulated physical enror is given by
8(x) e, (x) + 0.1e + 10e,(x),
(20)
where a large weight is used for (x) because the inequality constraints that it involves are very critical for the design. Among the three types of errors, e, ex() ) an implicit error, while ey ey(x) and ex(x are explicit errors.
18

---

B Extra Implementation Information
In this section, introduce extra implementation information for GEESE and the compared methods, in addition to what has been mentioned in the main text. In GEESE implementation, the latent vector z has the same dimension as the state x in problems 1 and 2, because the optimization is done directly on the latent vectors. In problem 3, the dimension of z is set be I, and transformed into a 30-dimensional vector x by the state generator. The number of the latent vector z used for sampling distribution of generators is set increasingly as d = 64, 128, 256 for problems 1, 2. and 3, due to the increasing dimension of the state space of the three problems. Although, the budget query number equals to 1000, because GEESE may query two times per iteration, thus, the maximum iteration number is smaller than 1000, which determined when the budget is used up.
For BOGP, its Bayesian optimization is implemented using the package [1]. The prior is set to be a Gaussian process, and its kernel is set Matern 5/2. The acquisition function is set be the upper confidence bound (UCB). The parameter kappa, which indicates how closed the next parameters are sampled, is tuned and the optimal value 2.5. The other hyperparameters are kept as default. Since Bayesian optimization only queries one state-error pair in each iteration, its maximum iteration number is equal to the maximum number of queries, i.e., 1000.
The other methods of GA, PSO. CMAES, ISRES, NSGA2, and UNSGA3 are implemented using the package pymoo (75]. For ISRES, optimize the offspring size, and finally apply 1/7 success rule to generate seven times more candidates than that in the current population in order to perform a sufficient search. The other parameters are kept default. Since these algorithms need query the whole population in each iteration, their maximum iteration number is thus much smaller than the query budget 1000. In the experiments, these algorithms are terminated when the maximum query number 1000 is reached.
To implement SVPEN 42 we use the default setting for problem 1. As for problems 2 and 3, to construct the state estimator and the error estimator for SVPEN, the same structures of the base neural networks and the exploitation generator used by GEESE are adopted, respectively. Also the same learning rate as used by GEESE is used for SVPEN, while the other settings are kept as default for problems 2 and 3. In each iteration, SVPEN queries three times the physical errors for simulating the exploitation, as well as the regional and global exploration. Thus, the maximum iteration number of SVPEN is set as 333 to match the query budget 1000.
All the methods are activated or initialized using the same set of N state-error pairs randomly sampled from predefined feasible region in the state space. For GEESE and SVPEN, these samples are used to train their surrogate error models, i.e., the base neural networks in GEESE and the error estimator in SVPEN, thus their batch size for training is also set N. For Bayesian optimization, these samples are used to construct the Gaussian process prior. For GA, PSO, ISRES, NSGA2, and UNSGA3, these samples are used as the initial population to start the search. The only special case is CMAES, as it does not need a set of samples to start the algorithm, but one sample. So we randomly select one state-error pair from the N pairs to activate its search.
For problem 3, we post-process the output of all the compared methods, in order to accommodate the element-wise inequality constraints in Eq. (19). by
1tc Ej x(1) (1- (1-x("))
(21)
As result, the magnitude of the element in {(P) is monotonically increasing, and the inequality constraints are naturally satisfied. But this can complicate the state search, as the elements are no longer independent. A balance between correlating the state elements and minimizing the accumulated physical error is needed. But in general, we have observed empirically that the above post- post-processing can accelerate the convergence for all the compared methods. One way to explain the effectivenees of this post-prec post-processing is that it forces the inequality constraints to hold, and this is hard for the optimization algorithms achieve their own.
C Extra Results
Varying Feasibility Threshold: In addition to the feasibility threshold of e = 0.075 as studied the main text, we test two other threshold values, including 0.05 representing a more challenging
19

---

Table 4: Performance comparison for two different values of feasibility threshold c, where the best is shown in bold while the second best is underlined for query times.
Problem
Problem
Problem
Threshold
Algorithm
State Dimennson::1
State Dimension:20
State Dimension:30
Failure times
Query times
Failure times
Query times
Failure times
Query times
BOGP GA
3.04 #0.83
78
849 #295.35
+2000.9
64 #0
0
65.92 #10.92
B
183.04 #287.80
PSO
0
64.00
199.92 1284.94
CMAES
0
12 +0
0
73,84 +25.81
3
127.29 +233.71
0.1
ISRES
65
108.52 141.36
10
203.30 1297.13
NSGA2
0
64 +0
D
70.40 199.20
B
189.04 +293.60
UNSGA3
0
28848--5 +16.3
7
177.52 +275.
SVPEN
82
932.51 =176.38 38
100
1000 =0
100
1000 #0
GEESE (ours)
2.34 #17.99
0
23.13 +17.99
0
35,58 163.82
BOGP
9.24 #3,97
I00
1000 30
16
227.63 4.66408
GA
64.00 +0
0
353.28 +1 +105.74 75
20
297 92 +363 45
PSO
64,00
1
157.84 1137.40
18
290.96 96 +373.65
0.05
CMAES
77. 56 +4.38
1
302.59 59 #1 156.
22
344.54 54 +363.
ISRES
193.00 =0
3
54 #241.22 1.22
19
313.69 +368.7 78
NSGA2
0
64.00 #0
0
352.00 #1 14.31
20
299.84 364.63
UNSGA3
64.00
368. 64 1002.85
310.72 +370.24
SVPEN
100
1000 =0
100
1000 10
100
1000 a0
GEESE (Ours)
20.20 +16.37
189.90 1164.96
81.26 #155.30
case with lower error tolerance, and c 0.1 representing a comparatively easier case with higher error tolerance. Results are reported in Table 4 In both cases, GEESE has failed the least times among all the compared methods and for all three problems studied. is worth to mention that, in most cases, GEESE has achieved zero failure, and a very small Niature 2 out of 100 in only one experiment when all the other methods have failed more than fifteen times. Also, this one experiment is the most challenging, solving the most complex problem 3 with the highest state dimension d = 30 and having the lowest error tolerance e = 0.05. In terms of query times, GEESE has always ranked among the top 2 most efficient methods for all the tested cases and problems, while the ranking of the other methods vary quite a lot. For instance, when = 0.05, BOGP performs the best for the easiest problem 1, but it performs the worst for the more difficult problem 2 where it has failed to find a feasible solution within the allowed query budget. In the most difficult experiment that studies problem 3 under e 0.05, GEESE requires much less query times and is significantly more efficient than the second most efficient method.
Varying Initial Sample Size:
In addition to the studied initial sample size N 64 in the main text, we further compare to more cases of N = 16 and N = 32 under = 0.05. The results are shown in Table 5 Still, GEESE has the least failure times in all experiments, which is important in remediating failed ML estimations. In terms of query times, GEESE still ranks among the top most efficient methods for the more complex problems 2 and 3. being the top I with significantly less query times for the most complex problem 3. However, GEESE does not show advantage in the simplest problem 1 with the lowest state dimension. It performs similarly to those top 2 methods under N = 32, e.g. 34 vs. 32 query times, while performs averagely when the initial sample size drops N 16. This is in way not surprising, because BOGP, GA, PSO, NSGA2 and UNSGA3 can easily explore the error distribution of low state dimensions. BOGP uses Gaussian process to construct accurate distribution of errors, while GA, PSO, NSGA2, and UNSGA3 sample sufficient samples in each iteration to sense the distribution of error in each iteration, and there is a high chance for them to find a good candidate in early iterations when the search space has a low dimension. However, the valuable samples are sparsely distributed into the higher dimensional space, and it is challenging for them to explore the error distribution and find the feasible states in the early iterations.
D GEESE Sensitivity Analysis
We conduct extra experiments to assess the hyperparameter sensitivity of GEESE using problem 1 under 0.05. The studied hyperparameters include the number of the base neural networks, the number Nir of the candidate states generated for exploitation, the learning rate for training the exploitation generator 1)T, and the early stopping threshold e, for training the base neural networks. The results are reported in Table[ It can be seen from the table that, although the performance varies versus different settings, the change is mild within an acceptable range. This makes convenient to tune the hyperparameters for GEESE.
20

---

Table Performance comparison under for two different sizes of initial samples, where the best is shown in bold while the second best is underlined for query times.
Problem
Problem 2
Problem
Initial Size
Algorithm
State Dimension:11
State Dimension:20
State Dimension:30
Failure times
Query times
Failure times
Query times
Failure times
Query times
BOGP
9.60 +3.89
100
1000
15
239.12
GA
0
32,00 =0
0
241 60 =71.75
21
=382.5
PSO
32,00
18
311.20 a:33, 45
14
283.28 #324.54
CMAES
0
77.56 14.38
1
321,01 4188.6
22
280.54 1363.18
32
ISRES
64.00 +0
3
416.24 +209 23
21
276.24 #386.75
NSGA2
32,00 +0
1
239.44 #150.26
22
262.88 =394.99
UNSGA3
0
32,00 +0
2
218.72 #136.53
22
260.64 #396.51
SVPEN
1000
100
1000
100
1000 a0
GEESE (Ours)
0
33.63 119.35
0
233.96 36,96666 +180.01
10
167.77 #284.31
BOGP
10.62 #5.53
100
1000 10
17
249.88 1372.99
GA
0
16.00 +0
43
657.04 +352.42
23
364.40 #373.30
PSO
0
32,00 1:
10
293.76 127 1.02
21
247.76 +392.87 87
CMAES
77.56 +4.38
333.49 #156.24
#350.
16
ISRES
0
17.00 +0
2
260. 50 +189.71
20
243.20 #392,70
NSGA2
32.00 =0
33
590 96 #355.93
25
377.20 =385.78
UNSGA3
0
32.00 :0
28
487.04 #360.22
28
408. 80 =397.77
SVPEN
100
1000 10
100
1000 10
100
1000 a0
GRESE(OUrS)
36.72 +22.52
0
248.26 #176.64
9
163.26 #279.34
Below we further discuss separately the effects different hyperparameters and analyze the reasons behind: (1) We experiment with three base network numbers L 2, 4, 8. It can be seen from Table 6 that there is a performance improvement as L increases terms of the required query times, but this is on the expense of consuming higher training cost. Thus, we choose the more balanced setting L = 4 as the default in our main experiments. (2) We test different candidate state numbers Nn = 1, 32, 64, 128 used for exploitation. Results show performance increase followed by decrease as Nir increases. Using a candidate set containing one single state is insufficient, while allowing set with too many candidate states can also harm the efficiency. This be caused by the approximation gap between the surrogate error model and the true physical evaluation. In our main experiments, we go with the setting of 64 for problem 1 as mentioned in Appendix B because it provides a proper balance between the exploitation performance and the overfitting risk. (3) We also examine different settings of the learning rate for training the exploitation generator, i.e., VIT lee 1e-2, le-3, Similarly, there is performance increase first but followed by decrease, in changing Nir- A larger learning rate can accelerate the learning of the exploitation generator and subsequently enable potentially faster search of the feasible state. But an overly high learning rate can also cause fluctuation around the local optimum, and this then consumes more query times. Although smaller learning rate can enable more guaranteed convergence to the local optimum, requires more iterations, thus more query times. (4) We experiment with three values of early stopping threshold, i.e., 1e-3, le first improve the efficiency but then reduce it, however without changing much the standard deviation. An inappropriate setting of the early stopping threshold can lead to base neural networks overfitting (or underfitting) to the actual data distribution, thus harm the performance.
1e-5. It can be seen from Table 6 that decreasing can
Table 6: Results of sensitivity Analysis, where better performance is highlighted in bold.
(I): Effect of Base Network Number
(2): Effect of Latent Vector Number
Base Network Number
Query times
Lafent vector number
Query times
Nn
72.56 #36.13
20.20 +16.37
Nir
28.21 #17.05
L 4
15.44 +13.86
Nir 64
20,20 #16.37
= 8
15.09 113.01
Nn
26.95 #14.12
(3); Effect of Learning rate for Exploration Generator
(4): Effect Early Stopping Threshold
Learning Rate
Query times 27. =9.28
Early stopping threshold
Query times
16
le
37.55+17.28
fr 1e-2
20.20=16.37
le-1
20.20 +16.37
Ph le-3
64.36 +44.50
Le
26.20 #15.45
21


Reviews:
[{"review_id": "HIR9HAfBHO", "reviewer": "Reviewer_Rpgy", "summary": "The article presents a new method of solving inverse problems using what they call is a grey box method. The introduce key concepts of using Generative methods to reduce the number of objective function invocations in an optimization problem. The presented results are very good and show a lot of promise for the techniques. Although there are a few points that are not addressed properly. They are mentioned in the sections below.\n\t\nAnother good thing with the method is that you can continue doing Exploration in Parallel, although this is not discussed in the paper. But on this point, I have other concerns that are mentioned in the Questions section.", "questions": "Line 207, if you are not training the generator then there it can be sampled from an appropriate distribution.  How random weights are going to help you when they are not being trained?", "limitations": "The authors have mentioned the limitations of complex systems with a large number of states. They have mentioned that they will work on this problem in the future. They will look into how they can solve the problem of extended training time and data requirements.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The methodology is explained in detail. The appendices are used adequately to describe the required concepts and proofs in detail. The code is provided in the supplementary material, but I could execute it because not much guidance was provided on how to execute the code. ", "weaknesses": "The method of explanation is not correct. In lines 126 to 130 authors have explained the algorithm briefly, which is not explained correctly. Important details are missing which creates misunderstanding. A better way would have been to introduce a complete sketch of the algorithm in the form of an algorithm that could be presented here. Then each part of the algorithm could be explained later, for example, Hybrid Neural Surrogate Error Models and Twin State Selection. Another way could be do first embed these fundamental concepts for the reader and then explain the algorithm in one go. Personally, I will recommend the first approach because that is iterative and reinforcing.\n\nFollowing are some language-related issues.\n- Line 83\n   A grey-box setting is thus more suitable in practice, where ones do not\n   [Correction]\n   A grey-box setting is thus more suitable in practice, where one does not\n- Line 141:\n   followed by the twin state selection for charactering the probability \n   [Correction]\n   followed by the twin state selection for characterizing the probability \n- Line 257\n   for evaluatation are explained \n   [Correction]\n   for evaluation are explained \n- Line 307\n   general, hile outperforming other methods\n   [Correction]\n   general, while outperforming other methods\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the insightful comments. We address the comments by grouping them into two categories: questions and discussion about weaknesses.\n### Replies to questions\n**Q1** Thank you  for  asking this interesting question. The purpose of the exploration generator  $\\mathbf{G}_R$  is to randomly sample a set of candidate states to select one state from by applying the criterion in Eq. (10). This is an improved selection strategy  over the exploration via disagreement (EVD) approach commonly used by RL (see references [57,58] as in the submitted paper). The difference is that EVD selects from the whole space, while ours selects from the candidate set sampled by $\\mathbf{G}_R$ to reduce computational cost. Theoretically, any probability distribution can be used as $\\mathbf{G}_R$. To encourage diversity,  different distributions are used for   different iterations  by adopting a parametric probability distribution function and varying its parameters at each iteration. In our design, we  use neural network to formulate such a  parametric distribution and varying its parameters by using random network weights. An alternative option is to use the  multivariate Gaussian distribution  and assign different  mean vectors and covariance matrices at different iterations.  In our practice, the neural network based distribution is sufficient to encourage diversity and select an effective exploration state.\nWe are happy to enrich the discussion of our exploration generator design in the revised manuscript.\n\n\n### Discussion about weaknesses\n **1** Thank you for the valuable suggestion. In lines 126 to 130, we started by explaining the commonly used iterative framework by classical black-box optimization techniques, instead of our proposed algorithm. It would be more clear to include a sketch of our algorithm after this.  We also prefer the first approach as suggested by the reviewer,  and will include an algorithm sketch in the revised manuscript and then expand on each algorithm part. We have prepared such a sketch and have attached it in the submitted pdf file as Algorithm 1.\n\n**2** Thank you for carefully reading our paper and helping improve the writing. We have made the corrections and polished the English in the revised manuscript.\t\n"}, {"review_id": "rdOVe24p6V", "reviewer": "Reviewer_Kiaj", "summary": "The paper is very good and interesting - and on a relatively novel topic that has so far recently seen very less attention in the AI community, and is only starting to see more attention with the recent questions on fairness and trust in AI models. The proposed GEESE algorithm is interesting, the foundation of the paper is mathematically sound, and this reviewer appreciates the idea of the authors to use the algorithm to build trust and confidence in traditional AI models.", "questions": "N/A", "limitations": "The limitations have been clearly addressed in the conclusions section/discussions of the paper.", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The contribution to the world of physics + AI combined is in itself very useful for the AI community. The major USP of the paper is its clear thought process on how in complex real-world engineering systems the error tolerance is often very low, and thereby, one can obviously not trust traditional optimisation techniques in ML. The idea of using physics-informed optimisation is appealing.", "weaknesses": "Only issue with the paper is its title - it is very generic. This reviewer would suggest the authors to make the title more specific (e.g. include the mention of ML models in the title or something similar to show that it pertains to utility in ML applications, as is the thought with NeurIPS!).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for appreciating our work and the very useful suggestion. Indeed the title should link better with the machine learning community. The new title will be **\"A physics-driven machine learning framework for correcting inverse estimation\"**."}, {"review_id": "WgBZJvjz0D", "reviewer": "Reviewer_EBsP", "summary": "This paper proposes a sample-efficient method for correcting failed states in surrogate inverse problems. This is achieved through error estimation of optimized surrogate states, where some computationally expensive errors are approximated with a neural network, while the simple errors are computed explicitly. Exploitation and exploration networks sample new states to verify errors for, finding feasible states that will not lead to failure. Several science and engineering applications are shown for the proposed method, and it outperforms the baselines for all tasks in query time and failure times.", "questions": "1) In line 99 for the related works paragraph on RL, is it correct to assume that there is no practical benefits of RL for inverse problems as opposed to the black box optimization mentioned in the paragraph above? At least there does not seem to be any advantages mentioned in this paragraph on RL for inverse problems. Is RL able to find better optima, but is just resource-intensive?\n\n2) Why is the exploration network using a generative model at all? Is the state space poorly parametrized, i.e., too high dimensional? Otherwise the maximization in Eq. 10 could just be performed over all states, and not the generative model of the states.\n\n3) Did the authors tune hyperparameters for the baselines as well for a fair comparison?\n\n4) In Line 285 the authors simplified the setting of X=Z, is it correct to assume there is no generative model in this case?", "limitations": "Limitations are clear, although understanding why the surrogate for implicit errors is necessary should be elaborated. This was introduced in a too vague manner.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Well written abstract, concise, showing core idea and significance in the field. The proposed method is well explained, and easy to understand. The authors did a thorough comparison with baselines, and ablation study to investigate the different components of the work.", "weaknesses": "1) Minor note: The introduction could be shorter, its purpose is very clear in that it leads to the contributions and methodology of the proposed work in the last paragraph, but there might be too much information leading up to that.\n\n2) The D-dimensional state space first mentioned in Line 151 lacks a practical introduction, the reader might find it hard to imagine what such a state space could look like. Is it a meshed airfoil shape, is it the 3D position and velocity vectors of some object, etc. This is mainly important since in Line 183 the authors explain the generative model for a candidate set of these states, which is then again more difficult to understand without knowing what exactly these states are. \n\n3) The implicit errors should be more thoroughly explained, it is unclear how \"expensive to collect\" is defined, and once again, a practical example of how these look like would be appreciated. Most important of all, it is unclear why these should be estimated with a surrogate in the first place. If it is due to computational complexity, an analysis should be done on how slow the computation is, and how much of a speedup the surrogate can offer, all the while showing the limitations of the surrogate in terms of generalizability, since the error estimation likely worsens when the explored states are far off from existing data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the insightful comments. We address the comments by grouping them into two categories: questions and discussion about weaknesses.\n### Replies to questions\n**1** Thank you for starting this interesting discussion. From our perspective, the biggest difference between RL and black-box optimization (BBO) is that the emphasis of RL is to train a policy/agent model that could be powerful for all scenarios, while BBO tries to search for the optimal solution for a given scenario. Such a  general policy/agent model requires more iterations for training, which is more data and time consuming than BBO. In our experiments, we have compared SVPEN [43], an algorithm inheriting the idea of RL, with BBO (see Table 1). The results show that it does query more data. Although RL is not efficient as an online optimization tool, it can train powerful models. One can use GEESE to improve further an estimate state obtained by RL.  In addition, RL can tackle very complex inverse problems via the existence of the policy model, one example is alpha-tensor[A], whereas BBO may be not suitable. We will enrich the related work paragraph on RL in the revised draft.\n\n**2** Thank you for raising this question. We would like to point out that, in most practice, the state space is factually decided by the application, rather than being parameterized as an algorithmic choice. The reason to use a generative model is to enable a maximization over a sampled set of states rather than the whole state space,  as we aim at finding a largely disagreed state instead of the most disagreed state. As we empirically observed during algorithm development,  the inclusion of the most disagreed state can cause instability in search process, and being slower. The goal of performing exploration over the whole space is to find the most disagreed state, for which the exploration via disagreement (EVD) approach used by RL [57,58] serves as a tool by putting effort in training the exploration generator. We have conducted ablation study comparing EVD with our approach, which corresponds to the Ablation Study (2) in Table 2. The results show that our approach performs better with fewer query times.  We will explain the above motivation for using an exploration generator in the revised draft.\n\n**3** Yes, we did. Specifically, we tuned the kappa of utility function in Bayesian optimization, the learning rate of SVPEN, and the success rule of ISRES.  As for the hyperparameters in GA-style algorithms, such as population size and offspring size, they are fixed to the batch size of GEESE, because it is unfair for GA to use a much larger population size than GEESE, which will exhaust the query budget in one run. In addition, problem 1 is directly inherited from the paper of SVPEN [43], so we took the default setting of SVPEN for that. We will include the information in Section B of supplementary material.\n\n**4** Thank you very much for noticing this potential misunderstanding that could be caused by the expression of $X=Z$, and we apologize for not being clear in line 285. It means that we set the latent space $Z$ as the state space $X$ without using any neural network to transform in between. We randomly sampled an initial state sets $X_{IT}^{0}$ containing $K$ states sampled from the state space. The exploitation state is directly optimized iteratively, e.g., by a gradient descent approach, based on  Eq. (7) (its slightly modified expression is shown in our submitted pdf file). The different states in $X_{IT}^{0}$ are used to initialize the optimization and result in $K$ solution states. The one with the smallest objective function value is selected as the exploitation state.\n\n### Discussion about weaknesses\n\n**1** We are happy to shorten the introduction in the revised draft.\n\n**2** When we generally introduce inverse problem and state in the beginning of the paper, we have given some practical examples of state in lines 21-23. But we agree that it will benefit the readers' understanding to use examples when explaining the method. As suggested, we will add the following text after the sentence in line 151: \"*An example of such a state space is a $D=2$ dimensional space, where the two dimensions correspond to the temperature and concentration states from spectroscopy.  Another example is a state space of $D=11$ dimensions with each state corresponding to a design parameter for aeroengine design, for which we provide more details in Section 4 and Section A of supplementary material.*\"\n\n**3** There are two types of implicit error that drive us to construct surrogate model to estimate them. As pointed by the reviewer, one type is the expensive error that takes time to calculate.  An example of this is a high-fidelity engine simulator in [B]   that takes two weeks to get a converged simulation. But, more importantly, a stronger drive for constructing surrogate models is the existence of those non-differentiable errors obtained from non-differentiable simulators, which mostly contain databases and maps. Examples of such simulators include those of spectroscopy [17] and of gas turbines [11].  The error gradient information is important for optimizing Eq.(6). Therefore, although there is a risk of obtaining inaccurate estimation, the surrogate model is used to estimate the errors and their gradient.  We do aim at improving the estimation accuracy in our design. For instance, we use ensemble learning to improve model robustness, and propose the twin state selection approach to query more useful states in order to improve the data quality for training the surrogate model. We will surely explain more on implicit errors and why estimating them in the revised draft, as suggested by the reviewer.\n\n[A] Fawzi, Alhussein, et al. \"Discovering faster matrix multiplication algorithms with reinforcement learning.\" Nature, 2022.\n\n[B] Claus, Russell W., and Scott Townsend. \"A review of high fidelity, gas turbine engine simulations.\",  2010."}, {"review_id": "3FLe9rRhOf", "reviewer": "Reviewer_aYMv", "summary": "This paper studies the problem of estimating states from observations (inverse problem). The problem is important since there are many applications such as engine design for aircraft. The current solution usually uses a black box simulator to simulate the observations given the estimated states from the model and compare them with the ground-truth observations to adjust the estimation of the state. This is based on the gradient-based method. However, the current state-of-the-art approach is time-consuming, in which it requires many quires and gradient update steps to find a good solution. As a result, this paper aims to increase the sample efficiency of such a method.\n\nThe proposed method is called \"Generative Exploitation and Exploration guided by hybrid Surrogate Error (GEESE)\". It consists of several parts: (1) a surrogate error model that consists of an ensemble of neural networks to provide a fast estimation of the physical error, and (2) a generative twin state selection algorithm that consists of two neutral networks for generating the state's distribution. The algorithm proceeds as follows: it first proposes an \"exploitation\" state that aims to search for a better solution that is near optimal, and then an \"exploration\" state that aims to explore other states that might lead to a better solution. The proposed method is tested in three real-world inverse problems and shows a competitive performance.", "questions": "1. In Table 1, there is a metric called query times. Could you please report the world-clock time in seconds? I am curious to see if these three problems simulator actually requires an hour/minute to run. \n\n2. How will the model perform if the size of the observation is in the order of thousand?  ", "limitations": "See the weakness section. Overall, I think this paper needs some further discussion.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The writing is very clear, and the notation is great.\n\n2.The method is simple and intuitive, and it shows good results based on Table 1.", "weaknesses": "1. The method requires the training data to train the neural networks to estimate the error. This would require a lot of computational cost and time to collect high-quality data.\n\n2. The proposed method essentially tries to learn a \"system dynamics model\" using neural networks and exploits this neural network to do optimization. As a result, you do not need to query the simulator many times.\n\n3. Following on the above point, this means that the quality estimation heavily relies on the performance of the hybrid surrogate model. How many datasets do you use? If I understand correctly, first you \"pre-train\" the model, and then use that for getting the results in Table 1? \n\n4. There is no explanation about what are the tasks presented in Table 1. I knew it states that they are in the appendix, but the reviewers are not required to read the appendix. So it would be nice to include some detail about the tasks.\n\n5. The observation dimension in the paper is quite low, compared to some prior works that estimate the state from image observations (https://proceedings.neurips.cc/paper_files/paper/2022/file/6d5e035724687454549b97d6c805dc84-Paper-Conference.pdf, https://arxiv.org/abs/2102.06794).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the insightful comments. We address the comments by grouping them into two categories: questions and discussion about weaknesses.\n### Replies to questions\n**1**  Thank you for the question. In practice, different types of simulators exist for the same problem. For instance, for problem 1, a  simulator mentioned in [A] with high fidelity takes 10.3 seconds $\\times$ 309 iterations  $\\approx$ 53 minutes to obtain a converged simulation, while another simulator in [B] with even higher fidelity takes two weeks to get a converged simulation. Since we aim at a research scenario that attempts to search a feasible state without querying too much data and without setting a high standard on data quality, we choose to use faster simulators with lower but reasonable fidelity in our experiments. For the three studied problems,  each simulator takes no more than five seconds to run. Since the computing time varies when changing simulators, we report the performance by query times. We will include such information in the revised draft for better clarification.\n\n[A] Zhang, Xiaobo, Zhanxue Wang, and Li Zhou. \"An integrated modeling approach for variable cycle engine performance analysis.\" Journal of the Franklin Institute 360.8 (2023): 5549-5563.\n\n[B] Claus, Russell W., and Scott Townsend. \"A review of high fidelity, gas turbine engine simulations.\" Proc. 27th Int. Congress of the Aeronautical Sciences, Nice, France, 19-24 September 2010. Bonn, Germany: International Council of the Aeronautical Sciences, 2010.\n\n**2** Thank you for raising this question. An observation with thousands of dimensions will not affect the calculation much.  As we mentioned in lines 118-121, and demonstrated in the original Figure 1, an observation, denoted by $\\mathbf{y}$, is used to calculate the error in GEESE and serves as a constant when correcting the state estimation. Its size only affects the error calculation, but not the optimization in state space. For instance, the choice of the distance function used to measure the difference between the ground truth observation $\\mathbf{y}$ and the simulated one $\\hat{\\mathbf{y}}$ using the estimated state $\\hat{\\mathbf{x}}$ should depend on the observation nature and size.\n\n### Discussion about weaknesses\n\n**1**  We agree with the reviewer that training data (state-error pairs in our study) is needed to train the surrogate error model (an ensemble neural network in our study). But our research objective is to reduce the cost of collecting high-quality training data, since we collect data online instead of having a ready dataset.  We collect a small amount of data to activate the training, and then incrementally collect as small amount of new data as possible at each iteration, aiming at effective online learning.  To avoid the use of a large readily collected data and to avoid high computational cost of data collection, GEESE uses a twin state selection approach to query the simulator at most twice in each iteration  (please refer to Sec.3.2). In experiments, for the studied problems, we only collect randomly 64 state-error pairs to activate the training (please refer to line 274), and by the time when a feasible state is found, the number of new state-error pairs incrementally collected is between 3-63  (see Table 1).\n\n**2**  Thank you for this comment. In GEESE, the surrogate model training (a learning process) and the optimal state search/exploration (a search process) are conducted in an alternating manner at every iteration. Such an incremental learning manner and an alternating nature do require to access the simulator many times. Specifically, unlike those conventional surrogate-model-based optimization mentioned in lines 85-98, the surrogate model in GEESE is not pre-trained using a readily collected large dataset, but trained from scratch  (please refer to sec.3.2), where, at each iteration, the states selected by the search process are used by the learning process to update the surrogate model. In addition, the surrogate model may provide unreliable estimations, which is critical for the applications of low error tolerance, as discussed in lines 95-98, so we also need to query the simulator to assess the quality of the corrected state estimation. Therefore, the simulator is necessarily queried multiple times.\n\n**3** We agree with the reviewer that the estimation quality heavily relies on the performance of the hybrid surrogate model. We would like to re-iterate that the surrogate model is not pre-trained via pre-collected datasets. Instead, the data is collected via online querying the simulator. As mentioned before, our training is performed incrementally and aims at using as few training data as possible, which, for the studied problems, include 64  randomly collected state-error pairs and another 3-63 carefully selected pairs.\n\n**4** Thank you for the useful suggestion. A task description will be incorporated after the first sentence of Section 4 in the revised draft.\n\n**5**  We are thankful to the reviewer for sharing these two papers. We would like to clarify that our research aims at reducing data usage through more effective online search strategies for optimization supported by a surrogate error model trained from scratch, and also aims at reducing data usage in surrogate model training by selecting more useful states to query. This is in contrast to the approaches reported in these two works, which are based on data-hungry, pre-trained models. As stated in our previous response under Question 2, the observation size is not of concern in the proposed approach. However, the state size matters, and this is discussed in lines 342-346, together with the potential limitations."}, {"review_id": "drcrDtAssv", "reviewer": "Reviewer_mU9t", "summary": "This paper builds on recent advances in black-box optimization for science and engineering inverse problems to find a satisfactory state with small physical error while decreasing the query times to the physical evaluation. The authors propose an error correction method GEESE, which is composed of a hybrid surrogate error model and a generative twin state selection approach. The hybrid surrogate error model is defined by both explicit and implicit errors, with the implicit error modeled as an ensemble of fully-connected neural networks. The generative twin state selection method adopts an exploration and exploitation approach, reducing the cost associated with physical error collection by not directly choosing states via a comprehensive space search. Experiments in real-world engineering inverse design settings demonstrate that GEESE decreases the number of queries needed to find satisfactory states with low error tolerance.", "questions": "Is there any theoretical evaluation between the estimated state that activates GEESE and a satisfactory state by GEESE, such as distance between them?", "limitations": "Yes, the limitations are discussed in the main text. ", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper introduces a novel correction algorithm, GEESE to find a qualified state within an error tolerance threshold after querying the physical evaluations as few times as possible. Overall architecture is novel, and a twin selection strategy, its key component that simultaneously propose a potentially near-optimal state and a potentially informative state, is effective to decrease the query times. \nThe experiments demonstrate that the proposed method outperforms the baselines in terms of both failure and query times. ", "weaknesses": "It is still unclear how existed ML model in ML-branch (in Figure 1) is incorporated in GEESE algorithm. Even though a state estimated by the existing ML-model is used for activating GEESE algorithm, what else is the ML-model used for?  Is the model also used to generate initial data $D_{0}$? (If so, does the performance of the ML model affect the failure time and query time in the experiments?) If ML-branch is used only for activation of GEESE branch, what is the reason of having ML-branch? In this case, as long as there is a pair of observation and estimated state (not necessarily being a state estimated by ML-model), GEESE is available for the pair. In other words, GEESE is applicable to no matter what ML-models or estimators we have.\n\n**Minor comments**\n\nline 149: Taking advantage of the robustness of nsemble learning [52, 53] … (typo)\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the insightful comments. We address the comments by grouping them into two categories: questions and discussion about weaknesses.\n### Replies to questions\nWe would like to sincerely thank the reviewer for this excellent question. Let us assume an observation vector $\\mathbf{y}$, and during training, its associated state vector $\\mathbf{x}$, and corresponding state vector estimate $\\hat{\\mathbf{x}}$. We would like to emphasize that during testing, when GEESE is deployed, the ground truth state vector $\\mathbf{x}$ for an observation $\\mathbf{y}$ is unknown, and thus, it is not possible to assess whether the estimated state $\\hat{\\mathbf{x}}$ is satisfactory or not by calculating a distance metric in state space, i.e., $d(\\mathbf{x}, \\hat{\\mathbf{x}})$, where $d$ is a suitable distance metric.\t\n\nTo remediate this, the error system including physical models and metrics is used to project the distance assessment in state space into the error space, i.e., $e(\\hat{\\mathbf{x}})=\\phi(d(\\hat{\\mathbf{x}}, \\mathbf{x}))$. Function $\\phi$ is the known physical model/metrics. By doing this, although we acknowledge the existence of $\\mathbf{x}$, it is masked in the error function. Notably, such a projection is nonlinear and depends on the physical process. Therefore, the distance in state space, although correlated to the error, is not directly proportional to the error. Theoretically, it is difficult to assess whether a state estimate is satisfactory or not via the direct measurement in state space. However, in practice, we can approximately assess it through an error tolerance threshold $\\epsilon$, which is set empirically. Given a  sufficiently small $\\epsilon$, when $e(\\hat{\\mathbf{x}})<\\epsilon$, we can assume $d(\\mathbf{x},\\hat{\\mathbf{x}})<k\\epsilon$ for some constant $k$. Therefore, we can say that the state estimate $\\hat{\\mathbf{x}}$ is satisfactory if satisfying the condition of $e(\\hat{\\mathbf{x}})<\\epsilon$. This thresholding approach is currently used to activate/continue the run of GEESE. \n\nThere are certainly deeper answers to explore your very interesting question, which we intend to pursue as further work, specifically, to perform geometric learning in state space, studying the geometry (e.g., distance and manifold structure) in state space that is compatible to the physical error.\t\n\n### Discussion about weaknesses\n**1** We agree with the reviewer in that GEESE can be applied within a general estimation framework, without the requirement of using a ML branch. As explained in the introduction part (Section 1), data-driven machine learning methods resulted in substantial advances in the solution of inverse problems (please check lines 25-27). However, these approaches lack in terms of reliability and trustworthiness, especially, for applications with low error tolerance. Indeed, this is the motivation for introducing GEESE as a state correction framework to enhance the reliability and trustworthiness of state estimations, while capitalizing on the efficiency of data-driven machine learning approaches. In regards to the query on the initial dataset $D_0$, this is randomly chosen as explained in line 3 of the algorithm pseudocode. Lastly, the reason for introducing the ML branch in Figure 1 was to emphasize that GEESE is a state correction framework for ML estimators. Following the reviewer's comment, we removed the branch in question, and revised Figure 1, as shown in the pdf attachment.\n\n**2** Thank you for pointing out the typo. We have corrected it in the revised manuscript."}]

Author rebuttal:
Dear reviewers, 

We are attaching a PDF file here for providing the information that we mentioned in separate rebuttals. Thank you very much.

Meta-review: This paper presents a new method for solving inverse problems using a grey box method. A sample-efficient technique is used for correcting failed states in surrogate inverse problems. This is achieved through error estimation of optimized surrogate states, where some computationally expensive errors are approximated with a neural network, while the simple errors are computed explicitly. The proposed method addresses a problem that receives little attention within the AI community. The paper is well-written, and technically strong, and the proposed technique is novel. The experiments support the claims made in the paper.
Decision: Accept (spotlight)