SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Ruiyuan Kang\*  
Bayanat AI  
Abu Dhabi, UAE  
ruiyuan.kang@bayanat.ai
Panos Liatsis  
Khalifa Univeristy  
Abu Dhabi, UAE  
panos.liatsis@ku.ac.ae
Tingting Mu  
University of Manchester  
Manchester, M13 9PL, United Kingdom  
tingting. mu@manchester.ac.uk
Dimitrios C. Kyritsis  
Khalifa Univeristy  
Abu Dhabi, UAE  
dimitrios.kyritsis@ku.ac.ae
When deploying machine learning estimators in science and engineering (SAE) domains, it is critical to avoid failed estimations that can have disastrous consequences, e.g., in aero engine design. This work focuses on detecting and correcting failed state estimations before adopting them in SAE inverse problems, by utilizing simulations and performance metrics guided by physical laws. We suggest to flag a machine learning estimation when its physical model error exceeds a feasible threshold, and propose a novel approach, GEESE, to correct it through optimization, aiming at delivering both low error and high efficiency. The key designs of GEESE include (1) a hybrid surrogate error model to provide fast error estimations to reduce simulation cost and to enable gradient based backpropagation of error feedback, and (2) two generative models to approximate the probability distributions of the candidate states for simulating the exploitation and exploration behaviours. All three models are constructed as neural networks. GEESE is tested on three real-world SAE inverse problems and compared to a number of state-of-the-art optimization/search approaches. Results show that it fails the least number of times in terms of finding a feasible state correction, and requires physical evaluations less frequently in general.
Many estimation problems in science and engineering (SAE) are fundamentally inverse problems, where the goal is to estimate the state  $\mathbf{x} \in \mathcal{X}$  of a system from its observation  $\mathbf{y} \in \mathcal{Y}$ . Examples include estimating the temperature state from the observed spectrum in combustion diagnostics [1], and discovering design parameters (state) of aero engine according to a group of performance parameters (observation) [2]. Traditional physics-driven inverse solvers are supported by rigorous physical laws, which vary depending on the application, e.g., the two-colour method for spectrum estimation [3], and cycle analysis for aero engine design [4]. Recent advances take advantage of machine learning (ML) techniques, constructing mapping functions  $F$  to directly estimate the state from the observation, i.e.,  $\hat{\mathbf{x}} = F(\mathbf{y})$  [5, 6, 7]. Such ML solutions are more straightforward to develop, moreover, efficient and easy to use. However, ML-based state estimates can sometimes be erroneous, while SAE applications have very low error tolerance. One can imagine the disastrous
*The work is done during Ruiyuan's PhD and Postdoc at Khalifa University
consequences of providing unqualified aero engine design parameters. Therefore, it is critical to detect and correct failed ML estimations before adopting them.
This leads to a special SAE requirement of evaluating the estimation correctness in the deployment process of an ML estimator. Since the ground truth state is unknown at this stage, indirect evaluation has to be performed. Such evaluations can be based on physical forward models and performance metrics [8, 9]. A common practice is to combine multiple evaluations to obtain an accumulated physical error, enforcing quality control from different aspects.
When the physical error exceeds a feasibility threshold, one has to remediate the concerned ML estimation. One practice for finding a better estimation is to directly minimize the physical error in state space [10]. This requires solving a black-box optimization problem, for which it is challenging to find its global optimum, iterative approaches are used to find a near-optimal solution [11, 12]. In each iteration, a set of states are selected to collect their physical errors, then error feedback is used to generate better state(s) until a near-optimal state is found. Physical error collection involves time-consuming simulations[13, 14], e.g., a spectrum simulation which, despite taking just several minutes for each run [15], can become costly if queried many times. Consequently, the optimization process becomes time-consuming. Therefore, in addition to searching a satisfactory state with as small as possible physical error, it is also vital to decrease the query times to the physical evaluation.
Our work herein is focused on developing an efficient algorithm for remediating the concerned ML estimation in deployment. We propose a novel correction algorithm, Generative Exploitation and Exploration guided by hybrid Surrogate Error (GEESE), building upon black-box optimization. It aims at finding a qualified state within an error tolerance threshold after querying the physical evaluations as few times as possible. The key design elements of GEESE include: (1) A hybrid surrogate error model, which comprises an ensemble of multiple base neural networks, to provide fast estimation of the physical error and to enable informative gradient-based backpropagation of error feedback in model training. (2) A generative twin state selection approach, which consists of two generative neural networks for characterizing the distributions of candidate states, to effectively simulate the exploitation and exploration behaviours. We conduct thorough experiments to test the proposed algorithm and compare it with a series of state-of-the-art optimization/search techniques, based on three real-world inverse problems. Results show that, among the compared methods, GEESE is able to find a qualified state after failing the least number of times and needing to query the physical evaluations less times.
Optimization in SAE: Development of SAE solutions often requires to formulate and solve optimization problems [16, 17, 18]. They are often black-box optimization due to the SAE nature. For instance, when the objective function is characterized through physical evaluations and solving partial differential equations (PDEs) [19], it is not given in a closed form. Typical black-box optimization techniques include Bayesian Optimization [20], Genetic Algorithm (GA) [21], and Particle Swarm Optimization (PSO) [22], etc. They often require a massive number of queries to the objective function in order to infer search directions for finding a near-optimal solution, which is time-consuming and expensive in SAE applications.
Instead, differentiable objective functions are constructed, and the problem is reduced to standard optimization, referred to as white-box optimization to be in contrast with black-box. A rich amount of well established solvers are developed for this, e.g., utilizing first-order and second-order gradient information [23]. Some recent developments use neural networks to optimize differentiable physical model evaluations, e.g., Optnet [24] and iterative neural networks [25]. However, physics-driven objective functions cannot always be formulated in a differential form, e.g., errors evaluated by the physical forward model in aero engine simulation, which is a mixture of database data, map information and PDEs [26]. A grey-box setting is thus more suitable in practice, where one does not unwrap the evaluations as a black box or oversimplify them as a white box, but a mixture of both.
Surrogate Model in Black-box Optimization: To reduce the cost of querying objective function values in black-box optimization, recent approaches construct surrogate models to obtain efficient and cheap estimation of the objective function. This practice has been used in SAE optimization, where the objective functions are mostly based on physical evaluations. The most popular technique for constructing surrogate models is machine learning (ML), including neural networks and Gaussian pro
cess models [27, 28, 29]. The associated surrogate model is then incorporated within an optimization process, guided by, for instance, GA and Bayesian optimization, which generate states and interact with it [30, 29], or neural networks that work with differentiable surrogate models [31, 32, 12]. To avoid overfitting, recent effort has been invested to develop surrogate models consistent with some pre-collected data, aiming at obtaining more reliable near-optimal solutions [33, 34, 35, 36, 37]. Nevertheless, there is no guarantee that a surrogate model can well approximate a physical model consistently. Indeed, this is the motivation for the proposed method, where surrogate models are used to speed up the querying process, while the decision in regards to the suitability of the solution is based on the actual physical evaluation.
Reinforcement Learning for Inverse Problems: In addition to black-box optimization based approaches, Reinforcement Learning (RL) [38, 39] serves as an alternative framework for solving inverse problems [40, 41, 42]. In an RL-based solution framework, physical evaluations are wrapped as a black-box environment outputting scalar reward, and the actions are the states to estimate according to the observation. The behaviour of the environment is simulated by training a world/critic model [43, 44], which is equivalent to a surrogate model of the physical evaluations. Different from black-box optimization based approaches, RL does not intend to search a feasible state estimation for the given observation, but to learn an authoritative agent/policy model [45, 46] to provide state estimations, while the policy training is guided by optimizing an accumulated scalar reward or error [47, 48]. Because of the desire of training a powerful policy model and the statistical nature of the reward, RL often requires many physical evaluations to collect diverse samples and validate training performance [49, 50]. This can be time-consuming when there is limited computing resource.
We firstly explain the notation convention: Ordinary letters, such as  $x$  or  $X$ , represent scalars or functions with scalar output. Bold letters, such as  $\mathbf{x}$  or  $\mathbf{X}$ , represent vectors or functions with vector output. The  $i$ -th element of  $\mathbf{x}$  is denoted by  $x_{i}$ , while the first  $k$  elements of  $\mathbf{x}$  by  $x_{1:k}$ . We use  $\| \mathbf{x} \|_1$  and  $\| \mathbf{x} \|_2$  to denote the dimension,  $l_{1}$ -norm and  $l_{2}$ -norm of the vector  $\mathbf{x}$ . An integer set is defined by  $[n] = \{1, 2, \dots, n\}$ .
Without loss of generality, an estimated state  $\hat{\mathbf{x}}$  is assessed by multiple physical models and/or metrics  $\{P_{i}\}_{i = 1}^{h}$ , resulting to an  $h$ -dimensional error vector, denoted by
$$
\mathbf {e} (\hat {\mathbf {x}}, \mathbf {y}) = \left[ E _ {P _ {1}} (\hat {\mathbf {x}}, \mathbf {y}), E _ {P _ {2}} (\hat {\mathbf {x}}, \mathbf {y}), \dots , E _ {P _ {h}} (\hat {\mathbf {x}}, \mathbf {y}) \right]. \tag {1}
$$
Each concerned ML estimation obtained from an observation  $\mathbf{y}$  is remediated independently, so  $\mathbf{y}$  acts as a constant in the algorithm, which enables simplifying the error notation to  $\mathbf{e}(\hat{\mathbf{x}})$  and  $E_{P_i}(\hat{\mathbf{x}})$ . A better state estimation is sought by minimizing the following accumulated physical error as
$$
\min  _ {\hat {\mathbf {x}} \in \mathcal {X}} e (\hat {\mathbf {x}}) = \sum_ {i = 1} ^ {h} w _ {i} E _ {P _ {i}} (\hat {\mathbf {x}}), \tag {2}
$$
where the error weights are primarily identified by domain experts according to the targeted SAE application. For our problem of interest, the goal is to find a state correction that is within a desired error tolerance, e.g.,  $e(\hat{\mathbf{x}})\leq \epsilon$  where  $\epsilon >0$  is a feasibility threshold, determined by domain experts. Thus it is not necessary to find a global optimal solution, instead a feasible solution suffices. A typical iterative framework for black-box optimization can be used for this. For instance, at each iteration  $t$ , a set of selected states  $\left\{\hat{\mathbf{x}}_i^{(t)}\right\}_{i = 1}^{n_t}$  are queried to collect their physical errors resulting in a set of state-error pairs  $\left\{\left(\hat{\mathbf{x}}_i^{(t)},\mathbf{e}_i\right)\right\}_{i = 1}^{n_t}$ . A state analysis is then performed according to the error feedback. In the next iteration, a new set of states  $\left\{\hat{\mathbf{x}}_i^{(t + 1)}\right\}_{i = 1}^{n_{t + 1}}$  are selected to query. This process is repeated until the feasible state  $\hat{\mathbf{x}}^*$  that satisfies  $e(\hat{\mathbf{x}}^{*})\leq \epsilon$  is found. When designing such a framework, the objective is to find a feasible state  $\hat{\mathbf{x}}^*$  by querying the physical errors as less times as possible because it is time-consuming to collect the errors.
We challenge the difficult setting of choosing only two states to query at each iteration. To ease the explanation, we first present a sketch of our proposed GEESE approach in Algorithm 1. It starts from an initial set of randomly selected and queried states  $\left\{\left(\hat{\mathbf{x}}_i^{(0)},\mathbf{e}_i\right)\right\}_{i = 1}^N$ . After this, at each
iteration, only two new states are selected to query based on a novel twin state selection approach that we propose, which selects a potentially near-optimal state for exploitation and a potentially informative state for exploration, resulting in  $\left(\hat{\mathbf{x}}_{\mathrm{IT}}^{(t)},\mathbf{e}_{\mathrm{IT}}^{(t)}\right)$  and  $\left(\hat{\mathbf{x}}_{\mathrm{R}}^{(t)},\mathbf{e}_{\mathrm{R}}^{(t)}\right)$ . The selection requires to perform error analysis for a large set of candidate states, involving both the errors and their gradients. To ease and enable such computation, we develop a differentiable surrogate error model  $\hat{e} (\mathbf{x},\mathbf{w})$  to rapidly approximate those error elements that are expensive to evaluate or in need of gradient calculation, and also provide informative gradient guidance with the assistance of error structure. The weights  $\mathbf{w}$  of the surrogate model are trained using the queried state-error pairs, which start as  $\left\{\left(\hat{\mathbf{x}}_i^{(0)},\mathbf{e}_i\right)\right\}_{i = 1}^N$  and then expand by including  $\left(\hat{\mathbf{x}}_{\mathrm{IT}}^{(t)},\mathbf{e}_{\mathrm{IT}}^{(t)}\right)$  and  $\left(\hat{\mathbf{x}}_{\mathrm{R}}^{(t)},\mathbf{e}_{\mathrm{R}}^{(t)}\right)$  at each iteration till the algorithm terminates by satisfying the feasibility condition  $e(\hat{\mathbf{x}}^{*})\leq \epsilon$ . Below, we first explain the process of constructing the surrogate model for error approximation, followed by the twin state selection for characterizing the probability distributions of the candidate states and collecting errors, and finally, the implementation of the complete algorithm.
Algorithm 1 Sketch of GEESE
1: Randomly select  $n_0$  states and query their physical errors to obtain  $D_0 = \left\{\hat{\mathbf{x}}_i^{(0)}, \mathbf{e}_i\right\}_{i=1}^{n_0}$ .
2: Train the surrogate error model  $\hat{e}\left(\mathbf{x},\mathbf{w}^{(0)}\right)$  with  $D_0$
3: for  $t \leq T$  do
4: Select a query state  $\hat{\mathbf{x}}_{\mathrm{IT}}^{(t)}$  by exploitation and collect the state-error pair  $\left(\hat{\mathbf{x}}_{\mathrm{IT}}^{(t)}, \mathbf{e}_{\mathrm{IT}}^{(t)}\right)$
5: Update the feasible state  $\hat{\mathbf{x}}^{*} = \hat{\mathbf{x}}_{\mathrm{IT}}^{(t)}$
6: Stop the algorithm if  $e(\hat{\mathbf{x}}^*) \leq \epsilon$
7: Select a query state  $\hat{\mathbf{x}}_{\mathrm{R}}^{(t)}$  by exploration and collect the state-error pair  $\left(\hat{\mathbf{x}}_{\mathrm{R}}^{(t)}, \mathbf{e}_{\mathrm{R}}^{(t)}\right)$
8: Expand the training data  $D_{t}$  using the two pairs  $\left(\hat{\mathbf{x}}_{\mathrm{IT}}^{(t)},\mathbf{e}_{\mathrm{IT}}^{(t)}\right)$  and  $\left(\hat{\mathbf{x}}_{\mathrm{R}}^{(t)},\mathbf{e}_{\mathrm{R}}^{(t)}\right)$
9: Keep training the surrogate error model using  $D_{t}$ , resulting in  $\hat{e} (\mathbf{x},\mathbf{w}^{(t)})$
0: end for
1: output  $\hat{\mathbf{x}}^{*}$
3.1 Hybrid Neural Surrogate Error Models
We start from an informal definition of implicit and explicit errors. Among the set of  $h$  error elements in Eq. (1), those that are expensive to collect or to perform gradient calculation are referred to as implicit errors. These can include cases where the system is too complicated and needs much more time to calculate the gradient than that of network backpropagation; or where the system is indifferentiable, such as the physical model of spectroscopy [15] and aeroengine [26] containing database or map. In addition to implicit errors, the remaining are explicit errors. We order these error elements so that the first  $k$  elements  $\{E_{P_i}(\hat{\mathbf{x}})\}_{i = 1}^k$  are implicit while the remaining  $\{E_{P_i}(\hat{\mathbf{x}})\}_{i = k + 1}^n$  are explicit. Our strategy is to develop a surrogate for each implicit error element, while directly calculate each explicit error.
Taking advantage of the robustness of ensemble learning [51, 52], we propose to estimate the implicit errors by an ensemble of multiple base neural networks. Each base neural network is fully connected with a mapping function  $\phi (\mathbf{x},\mathbf{w}):\mathcal{R}^{D}\times \mathcal{R}^{|{\bf w}|}\rightarrow \mathcal{R}^{k}$ , taking the  $D$ -dimensional state space  $\mathcal{R}^D$  as its input space, while returning the approximation of the  $k$  implicit errors by its  $k$  output neurons. An example of such a state space is a space of  $D = 2$  dimensions, where the two dimensions correspond to the temperature and concentration states from spectroscopy. Another example is a state space of  $D = 11$  dimensions with each state corresponding to a design parameter for aeroengine design, for which we provide more details in Section 4 and Appendix A. The network weights are stored in the vector  $\mathbf{w}$ . We train  $L$  individual base networks sharing the same architecture, while obtain the final prediction using an average combiner. As a result, given a state estimation  $\hat{\mathbf{x}}$ , the estimate of the implicit error vector is computed by
$$
\hat {\mathbf {e}} _ {\mathrm {i m}} \left(\hat {\mathbf {x}}, \left\{\mathbf {w} _ {i} \right\} _ {i = 1} ^ {L}\right) = \frac {1}{L} \sum_ {i = 1} ^ {L} \phi (\hat {\mathbf {x}}, \mathbf {w} _ {i}), \tag {3}
$$
and thus, the accumulated physical error is approximated by
$$
\hat {e} \left(\hat {\mathbf {x}}, \left\{\mathbf {w} _ {i} \right\} _ {i = 1} ^ {L}\right) = \underbrace {\sum_ {j = 1} ^ {k} w _ {j} \left(\frac {1}{L} \sum_ {i = 1} ^ {L} \phi_ {j} (\hat {\mathbf {x}} , \mathbf {w} _ {i})\right)} _ {\text {a p p r o x i m a t e d i m p l i c i t e r r o r}} + \underbrace {\sum_ {j = k + 1} ^ {h} w _ {j} E _ {P _ {j}} (\hat {\mathbf {x}})} _ {\text {t r u e e x p l i c i t e r r o r}}. \tag {4}
$$
We refer to Eq. (4) as a hybrid surrogate error model including both approximated and true error evaluation.
The weights of the base neural networks  $\{\mathbf{w}_i\}_{i=1}^L$  are trained using a set of collected state-error pairs, e.g.,  $D = \{(\hat{\mathbf{x}}_i, \mathbf{e}_i)\}_{i=1}^N$ . In our implementation, bootstrapping sampling [53] is adopted to train each base neural network independently, by minimizing a distance loss between the estimated and collected implicit errors, as
$$
\min  _ {\mathbf {w} _ {i}} \mathbb {E} _ {(\hat {\mathbf {x}}, \mathbf {e}) \sim D} [ \operatorname {d i s t} (\phi (\hat {\mathbf {x}}, \mathbf {w} _ {i}), \mathbf {e} _ {1: k}) ]. \tag {5}
$$
A typical example of the distance function is  $\mathrm{dist}(\hat{\mathbf{e}},\mathbf{e}) = \| \hat{\mathbf{e}} -\mathbf{e}\| _2^2$
Here, we choose to estimate each element of the implicit error vector, rather than to estimate a scalar value of the weighted error sum, because the structural information of the error vector can directly contribute in training through its associated gradient information. When estimating the weighted sum directly, it is in a way to restrict the training loss to a form loosely like  $\left( \hat{e}(\mathbf{w}) - \| \mathbf{e} \|_1 \right)^2$ , which negatively affects the information content of the gradient information. We have observed empirically that, the proposed individual error estimation leads to improvements in training the exploitation generator, compared to using the weighted error sum, see ablation study (1) in Table 2.
3.2 Twin State Selection
A selection strategy, i.e., twin state selection (TSS), for querying two individual states at each iteration is proposed, one for exploration and one for exploitation, respectively. The objective of TSS is to substantially reduce the cost associated with physical error collection. In turn, this translates to the formidable challenge of designing a selection process, which maximizes the informativeness of the associated physical error collection subject to minimizing query times. It is obviously impractical and inaccurate to adopt the naive approach of choosing directly one state by searching the whole space. Instead, we target at a two-folded task, researching (1) which candidate set of states to select from and (2) how to select.
By taking advantage of developments in generative AI, we construct generative neural networks to sample the candidate states. Specifically, we employ a latent variable  $\mathbf{z} \in \mathcal{R}^d$ , which follows a simple distribution, e.g., uniform distribution  $\mathbf{z} \sim U\left([-a,a]^d\right)$ , and a neural network  $\mathbf{G}(\mathbf{z},\boldsymbol{\theta}): \mathcal{R}^d \times \mathcal{R}^{|\boldsymbol{\theta}|} \to \mathcal{R}^D$ . The transformed distribution  $p\left(\mathbf{G}(\mathbf{z},\boldsymbol{\theta})\right)$  is then used to model the distribution of a candidate set. Thus, the task of candidate selection is transformed into determining the neural network weights  $\boldsymbol{\theta}$  for the generator  $\mathbf{G}$ .
In general, exploitation attempts to select states close to the optimal one, whereas exploration attempts to select more informative states to enhance the error estimation. There are various ways to simulate the exploitation and exploration behaviours. For instance, in conventional black-box optimization, e.g., Bayesian optimization and GA, exploitation and exploration are integrated within a single state selection process [54], while in reinforcement learning, a balance trade-off approach is pursued [55, 39]. Our method treats them as two separate tasks with distinct strategies for constructing generators and selecting states.
ExploITation: To simulate the exploitation behaviour, the exploitation generator  $\mathbf{G}_{\mathrm{IT}}$  is trained at each iteration by minimizing the expectation of the physical error estimate, using the hybrid surrogate error model
$$
\boldsymbol {\theta} _ {\mathbf {G} _ {\mathrm {I I}}} ^ {(t)} = \arg \min  _ {\boldsymbol {\theta} \in \mathcal {R} ^ {d}} \mathbb {E} _ {\mathbf {z} \sim U ([ - a, a ] ^ {d})} \left[ \hat {e} \left(\mathbf {G} _ {\mathrm {I I}} (\mathbf {z}, \boldsymbol {\theta}), \left\{\mathbf {w} _ {i} ^ {(t - 1)} \right\} _ {i = 1} ^ {L}\right) \right], \tag {6}
$$
where the base networks from the last iteration are used and we add the subscript  $t - 1$  to the weights of the error network for emphasizing. Finally, among the candidates generated by  $\mathbf{G}_{\mathrm{IT}}$  with its trained
weights  $\theta_{\mathbf{G}_{\mathrm{IT}}}^{(t)}$  , we select the following state
$$
\hat {\mathbf {x}} _ {\mathrm {I T}} ^ {(t)} = \arg \min  _ {\hat {\mathbf {x}} \sim p (\hat {\mathbf {x}} | \boldsymbol {\theta} _ {\mathbf {G} _ {\mathrm {I T}}} ^ {(t)})} \hat {e} (\hat {\mathbf {x}}, \left\{\mathbf {w} _ {i} ^ {(t - 1)} \right\} _ {i = 1} ^ {L}), \tag {7}
$$
to query its physical error by Eq. (1), resulting in the state-error pair  $\left(\hat{\mathbf{x}}_{\mathrm{IT}}^{(t)},\mathbf{e}_{\mathrm{IT}}^{(t)}\right)$ . If the queried error is less than the feasibility threshold, i.e.,  $\mathbf{e}_{\mathrm{IT}}^{(t)}\leq \epsilon$ , this selected state is considered acceptable and the iteration is terminated. Otherwise, it is used to keep improving the training of the surrogate error model in the next iteration.