Epidemiological Model Calibration with Bayesian Decision-Making

Authors: puhua  niu
byung-jun  yoon
xiaoning  qian
Venue: IEEE_org_EMBS_BHI_2024_Conference
Published: 2024-06-29 00:06:26

Abstract: In this study, we focus on developing efficient calibration methods via Bayesian decision-making for the family of compartmental epidemiological models. The existing calibration methods usually assume the compartmental model is *cheap* in terms of its output and gradient evaluation, which may not hold in practice when extending them to more general settings. Therefore,  we introduce model calibration methods based on a ``graybox'' Bayesian optimization (BO) scheme, more efficient calibration for general epidemiological models.  This approach uses Gaussian processes as a surrogate to the expensive model, and leverages the functional structure of the compartmental model to enhance calibration performance. Additionally, we develop model calibration methods via a decoupled decision-making strategy for BO, which further exploits the decomposable nature of the functional structure. The calibration efficiencies of the multiple proposed schemes are evaluated based on various data generated by a compartmental model mimicking real-world epidemic processes. Experimental results demonstrate that our proposed graybox variants of BO schemes can further improve the calibration performance measured by the logarithm of mean square errors and achieve faster performance convergence in terms of BO iterations. We anticipate that the proposed calibration methods can be extended to enable fast calibration of more complex epidemiological models, such as the agent-based models.

Main paper: 
 i. introduction: 
 Pandemics like the recent coronavirus disease (COVID- 19) have shown enormous impacts on public health worldwide. The development of computational epidemiological models is crucial for gaining better quantitative understanding of the disease spread and enabling swift decision-making to design effective mitigation strategies. In fact, well-calibrated epidemiological models can serve as a useful guide for forecasting and quantifying the epidemic risk and implementing effective public health measures to control the spread of epidemic diseases [2] , [8] .Compartmental models, such as the the commonly adopted SIR (Susceptible-Infectious-Recovered) model, constitute an important family of population-based epidemiological models. Models in this family take the form of Ordinary Differential Equation (ODE) systems that capture the general trends and dynamics in the subpopulations represented by different compartments. In this work, we focus on this family of models while considering a new "Quarantined" state represented by an additional compartment. This new state aims at better mimicking the dynamic trajectories of the epidemic spreads, typical in airborne diseases such as seasonal flu and COVID-19, resulting in a four-compartment SIQR model [23] .To accurately model real-world physical processes, it is crucial for computer models to fine-tune their model parameters based on observed data to capture inherent attributes of the physical processes, which can not be directly or easily measured by means of physical experiments. For instance, material properties, such as porosity and permeability are important computer inputs in computational material simulations, which cannot be measured directly in physical experiments. In the applied mathematics and computational science literature, the methods used to identify those parameters are called model calibration techniques and the resulting parameters are called calibration parameters [20] , [25] . The basic idea of calibration is to find the combination of the calibration parameters, under which the simulated computer outputs align with the observed physical data. A well-calibrated SIQR model may allow people to make more accurate predictions of pandemic risk and enable them to make better-informed decisions on how to mitigate.The existing method often assumes that the computer model is cheap [30] . This means that: the explicit form of the optimization objective f (x) for calibration parameters x is known and we can evaluate the first-or second-order derivatives of f (x) with respect to (w.r.t.) x. For an ODE system, deriving the explicit form of f (x) and ∇ x f (x) can be infeasible when the ODE is non-linear. Recent advances show that sensitivity analysis allows us to evaluate the gradient ∇ x f (x) and perform the gradient-descent optimization method for the estimation of x [12] , [18] . Correspondingly, physics-informed machine learning [19] is drawing increasing attention as it integrates data and mathematical models into deep neural networks or other regression models by enforcing fundamental physical laws.In real-world scenarios when calibrating a computer simulation software, we are usually blind to the explicit forms of both f (x) and ∇ x f (x) and we may be able to access only the output of f (x). We classify this type of computer models as expensive. Hence, it is valuable to pay attention to the calibration problem for such expensive computer models. Bayesian Optimization (BO) is one conventional approach for such scenario when evaluating f (x) is expensive, where we use a surrogate model to approximate f (x) and the optimization process is sequential. In each BO round, we make a Bayesian decision so that some expected utility is maximized and then the corresponding output is queried from the computer model based on this decision to refine the surrogate model to guide the optimization of the objective function f (x) [14] , [17] .This paper concentrates specifically on the calibration of the SIQR model based on BO and is organized as follows. First, we explain the configurations of the SIQR model and the model calibration setup. Then we introduce our calibration methods based on a "graybox BO" approach, instead of the usual "blackbox BO", where the experts' prior knowledge about the computer models is integrated into the BO formulation to improve its optimization performance. Finally, we explain how the graybox BO works for decoupled decisionmaking, accompanied by a new acquisition function. In our experiments, we evaluate the performance of our proposed graybox BO-based model calibration methods and compare them with traditional blackbox BO-based implementations to demonstrate the benefits of integrating prior model knowledge into the calibration process.In this model, people in a given population are classified into four compartments: Susceptible (S), Infectious (I), Recovered (R), and Quarantined (Q). Those who have not been infected are considered susceptible. Transmission occurs between susceptible and infected individuals. The number of symptomatic patients, equivalent to the number of infected individuals, decreases through treatment and/or quarantine. Recovered individuals are assumed to remain immune to further infection. Therefore, all people will be in the recovered group after an adequate period. The dynamics of the SIQR model are given by the following set of ordinary differential equations (ODEs) [23] :EQUATIONwhere S(t), I(t), Q(t) and R(t) ∈ [0, N] are the populations of compartments 'S', 'I', 'Q' and 'R' respectively.It can be verified that the ODEs guarantee the conservation of the population, S(t)+I(t)+Q(t)+R(t) = N . A schematic illustration of such an SIQR model is shown in Figure 1 . We note that there are many compartmental epidemiological models based on different extensions of the original SIR model [10] , [24] . While we focus on SIQR model calibration in this paper, the presented model calibration methods can be applied to all these variants in a straightforward manner. More importantly, calibration of more complicated epidemiology models based on agent-based models (ABMs) can also leverage our proposed methods if their critical parameters can be identified, where calibration efficiency is even more important due to their significantly higher computational complexity compared to the ODE-based compartmental models [1] , [4] , [9] .Most of the existing calibration methods [11] , [13] , [26] , [28] , [29] , [31] follow or adapt from the Bayesian approach pioneered by Kennedy and O'Hagan (KOH) [20] . In practice, we can only collect epidemiological data in the occurrence of an epidemic. In other words, we can observe the population's dynamic trajectory only once, making model calibration more challenging than in other statistical inference or machine learning tasks. Supposing one such trajectory with time length T are observed, we denote the observed epidemiological data as{d t = (d t 1 , ..., d t 4 )} T t=1and make the following assumption:EQUATION)where ζ is the unknown ground-truth underlying epidemic process, η is a computer model (e.g., SIQR model) that we would like to calibrate to simulate the behavior of ζ and fit the observed epidemiological data, x ∈ R D is the D-dimensional calibration parameter vector of η, and ∼ N(0, Diag(1)) represents the observation noise. In this work, R(t;  x)) corresponds to the output of the SIQR model with x representing parameters of rate functions {β(t; x), γ(t; x), δ(x), γ(x)}. By our assumption, d t |η t (x) ∼ N(η t (x), 1), and the Maximum Log-likelihood Estimation (MLE) of x corresponds to [28] , [30] :η t (x) = (η t 1 (x), η t 2 (x), η t 3 (x)), η t 4 (x)) = (S(t; x), I(t; x), Q(t; x),EQUATION)where g denotes the likelihood function as the negative Square Error (SE) function based on the Gaussian assumption. For notation compactness, we denote g(η t (x)) as f t (x). The goal of model calibration is the maximization of the corresponding negative Mean Square Error (MSE), t f t (x). III. METHODOLOGY Instead of following the traditional KOH Bayesian model calibration scheme [20] , we resort to a new model calibration framework leveraging Bayesian Optimization (BO) [17] . Under this framework, model calibration is formulated into the optimization of the correspondingly designed utility function, called acquisition function. In BO, Gaussian Processes (GPs) have been typically used as cheap surrogates of f (x) 1 . In this way, we circumvent the difficulties in optimizing f (x) directly. To be more specific, the standard BO approach treats f (x) as a blackbox function and starts modeling it by a set of GP prior distributions, N (μ(x), Σ(x)). Then, it iteratively chooses the next point at which to evaluate f with the procedure described as follows.Q(t) R(t) I(t) λ δ β(t) γGiven n observations of the objective function customized for model calibration, f (x 1 ), . . . , f(x n ), we first infer the posterior distribution, N (μ n (x), Σ n (x)), which conditions on {f (x l ), x l } n l=1 . Then, this posterior distribution is used to compute an acquisition function. Finally, we make a Bayesian decision that chooses the next point to be evaluated, x n+1 , as the point that maximizes this acquisition function. After N iteration, the candidates x best = argmax x μ N (x) is the calibrated parameter choice.It is clear that given the simulated dynamics η(x), evaluation of the model calibration performance metric g is cheap and does not need to be approximated. Instead of directly modeling f (x) as a blackbox objective function as typically done in BO, for SIQR model calibration, we here first use a set of GPs, denoted by y(x), as the surrogate to η(x). With that, we then model f (x) as a graybox objective, by a Composite Function (CF) [5] :EQUATION)Spared from the modeling of g, the model calibration performance can be expected to be further improved. Furthermore, as shown in Figure 1 , the workflow of the SIQR model encompasses a set of input-output structures between functions of each compartment. For example, infectious population 'I' are exclusively transferred from susceptible population 'S', and individuals in the recovery compartment 'R' only come from compartments 'I' and 'Q'. Therefore, one may expect that leveraging these functional structures may lead to improved calibration performance. Integrating these relationships into the BO method gives rise to a Function Network (FN), G(V, E), as shown in Figure 2 . In the graph, the set of nodes V = {v|y 1 , . . . , y 4 , x, g} denotes four GP sets associated with the four compartments 'S', 'I', 'Q' and 'R' respectively, calibration parameters x, and performance metric g. The graph also contains a set of edges E = {(v → v )|v, v ∈ V}, where (v → v ) means node v takes the output of nodes v as its input [6] . Suppose we already have made queries from the computer model η by simulating trajectories of subpopulations in each compartment at different settings and obtain D n = ŷl = η(x l ), x l n l=1 at n calibration parameter vectors. We have:EQUATIONwhere P a i = {y j : (y j → y i ) ∈ E} denotes the GPs that are the parent nodes of y i in graph G, P n (y i (x, P a i )) = P y i (x, P a i )|ŷ 1:n (x 1:n ) denotes the posterior distribution of GP y i with mean m i n and variance σ i n . Given D n , we choose the GP prior P y 1:n i (x 1:n , P a 1:n i ) to be multivariate normal with a Matérn covariance kernel and a constant mean, whose hyper-parameters are obtained by MLE over P (ŷ 1:n i (x 1:n , P a1:n i )).Then we infer the posterior from P (ŷ 1:n i (x 1:n , P a 1:n i ), y i (x, P a i )) [15] . Taking the Knowledge Gradient (KG) [15] , [16] , a typical formulation of acquisition functions as an example, we define the graybox acquisition function according to the FN as:α n (x) = E Pn(y(x)) u * n+1 (y, x) -u * n (6)whereu * n = max x u n (x), u n (x) = E Pn(y(x)) [g(y)], u *n+1 (y(x)) = max x u n+1 (x ; y(x)), and u n+1 (x ; y(x)) = E Pn+1(y (x );y(x)) [g(y )]. On the right-hand side of the formula above, the second term is the largest expected metric value based on the current GPs, and the first term is the largest expected metric value if the GPs are further conditioned on predicted data (y(x), x). In comparison, the expectation E Pn [y] and E Pn+1 [y] are computed in the original KG. Due to the maximization operator within the expectation, neither the original nor graybox versions of KG can be computed explicitly. Thus, we resort to Monte Carlo estimation [3] of the expectations and the reparametrization trick [21] , so that α is an unbiased estimation of α and ∇ x α is feasible. To be more specific, we estimate the acquisition function via:EQUATIONwhere ˆ i denotes a sample of i ∼ N(0, 1), K = 8, and L = 128. The inference of P n mainly takes account of the computation cost, which is about O(n 3 ) and does not scale up with the dimension of GPs' input (i.e. x or [x, P a i ]) [27] . Thus, the computation complexity of the original and graybox versions of KG are similar. The workflow of our calibration method based on graybox BO is illustrated in Figure 3 with the pseudo-code summarized in Algorithm 1.In practice, epidemiological data d from the unknown epidemic process ζ may not be observed at synchronized time points. In other words, some observations at the corresponding time point may be missing in practice. For example, when d = {∅, d 2 , d 3 , d 4 }, where d 1 is missing in the observed data, we may only be able to compute g(y(x)) = 1 T i =1 (y i (x)-d i ) 2 .In such a scenario of decoupled observations, the function network organization of GPs allows us to infer the function form y 1 (x) by leveraging the ground-truth functional dependency represented by G. To be more specific, where v ⊥ v indicates that v and v are statistically dependent so that P (v|v ) = P (v) for any two random variables v and v [22] . This mimics the functional dependency h(•) ⊥ h (•), which means that a perturbation in function h(•) will affect the solution space of function h (•); An g = {v|(v → . . . → g) ∈ G}, denoting the ancestor nodes of g with a directed path starting at the ancestor node and ending at g. The corresponding graph with d = {∅, d 2 , d 3 , d 4 } will have the edge (y i → g) removed, while g ⊥ y 1 still holds. Moreover, it is reasonable to assume that the approximation of the computer model's output {η 1 (x), . . . , η 4 (x)} by the four GPs varies in computational complexity. The reason is that the complexities of the chosen function forms of {η 1 , . . . , η 4 } are different. Therefore, some GPs may need more queried data from the computer model than others. Taking into account of the decomposable structure of P n (y(x)), we extend the graybox acquisition function by decoupled decision-making and define the acquisition function as:EQUATIONEQUATIONwhere z ∈ {0, 1} 4 and P n+1 (y (x ); z, y(x)) is defined as:4 i=1P n+1 (y i (x , P a i ); y i (x, P a i )) zi P n (y i (x , P a i )) 1-zi .(10) New candidates x n+1 is found by argmax x,z α n (x, z). When z = 1, P n+1 (y (x ); z, y(x)) = P n+1 (y (x ); y(x)). The proposed objective can be seen as a generalization of the KG objective, which measures expected improvement when GPs are conditioned on predicted data (y(x), x). Here we further accommodate the case when a subset of GPs is conditioned on predicted data. Thus, candidates x n+1 with higher expected improvement can be expected by the proposed objective.Due to the combinatorial nature of z, it is hard to optimize w.r.t. z directly. The most naive approach is to optimize α n w.r.t. x under every possible choice of z, and the resulting computational cost is 2 |z| times that of the non-decoupled version. For the SIQR model in this paper, |z| = 4 and the increased cost is acceptable. Besides, the non-decoupled version is just a special case of the decoupled version, where z = 1. Therefore, we can accelerate the computation by selecting a subset. In this paper, we choose {(z :z 1 = 1, z =1 = 0), . . . , (z : z |z| = 1, z =|z| = 0)} ∪ {z = 1}, resulting |z| + 1 times computation cost of the non-decoupled version. Algorithm 1 Model Calibration Workflow Require: Number of iteration N , Computer model η(x) (X, Y ) ← (X init , {η(x)} x∈Xinit ) //X init is a randomly initialized set of calibration parameter vectors. for n = {0, . . . , N -1} do D n ← (X, Y ) P n (y i (x)) ← N (μ i n (x), σ i n (x)) x n+1 ← argmax x αn (x). (X, Y ) ← X ∪ {x n+1 }, Y ∪ {η(x n+1 )} end for x best ← argmax x ûN (x)In this section, we conduct experiments to evaluate and compare the performances of model calibration using various Bayesian Optimization (BO) methods. We consider different setups of observed epidemiological data generated from simulated "ground-truth" models. In our experiments, both the ground-truth model ζ and the computer model to calibrate η are SIQR models. Once a trial of epidemiological data d is simulated, the ground-truth model is no longer accessible. It is also assumed that we are only accessible to computer models' outputs. The goal is to validate the effectiveness of our graybox BO methods for model calibration in expensive scenarios and investigate how our proposed acquisition functions can improve the performance of graybox BO-based model calibration.To generate data based on the ground-truth SIQR model ζ, we set β * (t) = 0.9I(t), λ * (t) = 0.1I(t), δ * = 0.2, γ * = 0.2. The model to calibrate, η, takes the same form with undetermined parameters x, that is, δ(t, x) = x 1 I(t), λ(t, x) = x 2 I(t), δ = x 3 , and γ = x 4 . The initial conditions for both ζ and η are: S(0) = 0.99 * N , I(0) = 0.01 * N , R(0) = 0 and Q(0) = 0. We consider two scenarios representing the gap between the ground-truth model and the computer model to calibrate. The first way is to add noise to the ground-truth observations d. The second is to have different setups of η from those of ζ. In our experiments, we set λ * (t) in ζ as a non-linear function, that is, λ * (t) = log([I(t), S(t), R(t)] [0.3, 0.06, 0.12] + 1), while λ(t) in η still takes the linear form mentioned above. The corresponding plots for the 30-day simulations are shown in Figure 4 .We compare calibration performances based on the following BO methods: (i) the standard Expected Improvement (EI) in the blackbox BO framework without integrating functional structures, (ii) the standard Knowledge Gradient (KG), (iii) the Knowledge Gradient with composite function (KG-CF) , (iv) Knowledge Gradient with function network (KG-FN), and (v) the Decoupled Knowledge Gradient with composite function (DG-CF). All algorithms have been implemented based on the BOTorch package [7] while the latter three are all graybox BO variants. In all our experiments, we have run each calibration method five times from the different random seeds, where each run starts with an initial dataset from the computer model,D 0 = ŷl = η(x l ), x l 2D+1 l=1 , where x l is randomly selected in [0, 1] D .Experiments under all three ground-truth models are performed in complete-observation and incomplete-observation setups. In the latter setup, observations of compartment 'S' are assumed to be unknown, so d 1 is not involved with the calibration procedure. This helps to investigate whether the function network organization of GPs allows more reliable model calibration under incomplete (decoupled) observation data d by leveraging the ground-truth functional dependency.The experimental results under the complete-observation setting are shown in Figure 5 . In all these experiments, it can be observed that our graybox BO methods, including KG-CF, perform significantly better than the blackbox BO methods, EI and KG. This demonstrates that integrating knowledge of g does help improve the calibration performance. The performance of KG-FN is slightly worse than KG-CF. This phenomenon can be ascribed to the following reasons. With complete observations, all the updating of GPs, {y 1 , . . . , y 4 } that approximate {η 1 , . . . , η 4 } are well-informed, making the information propagating via introducing statistical dependency less important. Besides, the function network organization of GPs make the acquisition estimator α more challenging to calculate reliably. As shown in (7) , inputs of each GP include not only x but also random samples of P a, which makes the estimated α unstable due to the stochastic nature of inputs, and consequently increases the difficulty of finding the optimal point of α(x) in each BO iteration. Last but not least, our proposed DG-CF performs better than KG-CF. This validates that the decoupled acquisition can better inform decisionmaking by conditioning on the predicted data of a subset of GPs, rather than the whole set of GPs.The experimental results under the incomplete-observation setting are shown in Figure 6 . Again, our graybox BO methods perform significantly better than the blackbox BO methods. It should be noted that KG-FN can outperform KG-CF when the observations are noisy. This shows that integrating functional dependency does help to inform the calibration process, while the performance gain is not significant due to the accompanied optimization difficulty that we mentioned above. Finally, our proposed DG-CF performs better than KG-CF, validating again the advantages of the decoupled acquisition function. In Figure 7 , We also visualize population fraction trajectories generated from the computer models under the calibrated parameters derived by all methods in the setting with noisy and incomplete observations. It can be observed that all methods can provide non-trivial calibration results, showing that our BO-based calibration methods have a promising potential for epidemiological dynamic model calibration, when the computed model is expensive. Among all the BO variants, graybox BO variants KG-FN and DG-CF render the mean trajectories with smaller bias to the ground-truth ones as well as smaller variance, compared to KG-CF, EI, and KG. This again confirms that utilizing the functional dependency structure and decoupled acquisition functions can help better inform decision-making during BO iterations, achieving higher sample efficiency.In this work, we have proposed epidemiological model calibration methods based on the BO framework. To further improve calibration performance, we have formulated the calibration problem as a customized graybox BO task, where expert knowledge about functional dependency and calibration performance metric function is integrated into the new acquisition function. Furthermore, we have proposed a decoupled acquisition function, which further exploits the decomposable nature of the functional structure. We performed experiments under three types of ground-truth models and two types of observation data to validate the model calibration performance of our proposed BO-based methods. The experimental results have demonstrated the efficacy of our BO-based strategies with different variants for enabling the calibration of expensive computer models. Within a small number of BO iterations (≤ 50), the proposed graybox BO strategies can achieve good performance, which is measured by logarithm of MSEs, and faster convergence in terms of the number of BO iterations than strategies based on standard BO. While the experimental results have shown that utilizing groundtruth functional dependency can help the calibration process, the resulting formulation of the acquisition function can lead to optimization difficulties in practice, which may impair the overall calibration performance. Thus, future work will focus on further investigation of the function network organization of GPs in BO to achieve better performance. Besides, in the current work, the function network is limited to a single system, where each node corresponds to an element within that system. In a more general case, there can be multiple sub-systems interacting with one another, with each node in the network representing an entire sub-system. Our future investigation will consider extending the proposed calibration strategies to accommodate this more general case. Finally, agent-based epidemiological models have a much larger number of critical parameters than compartmental models. Along with their significant computational complexity, the underlying functional dependency are also more complex, making their calibration tasks more challenging. Efficient calibration of ABMs for epidemiological modeling may require developing and validating different approximate solutions for BO acquisition functions, which remains an open problem for future research.This work was supported by the U.S. Department of Energy (DOE) Office of Science under Award KJ0403010/FWP CC132.In the following text, we will consider the maximization of the acquisition function f t (x) instead of t f t (x) for notation compactness, and omit the subscript t, where it does not introduce ambiguity. 
 
 Appendix:  



Reviews: review: title: title: Accept
paper_summary: paper_summary: This paper examined model calibration methods based on a “gray box” Bayesian optimization (BO) scheme, which leverages the functional structure of the compartmental epidemiological model to improve calibration performance.
strength_weakness: strength_weakness: None
questions: questions: None
title: title: Acceptable after revision.
paper_summary: paper_summary: This research work has presented and investigated an epidemiological model calibration method based on the bayesian optimization (BO) framework. The results demonstrate the efficacy of the BO strategies with different variants for calibrating expensive computer models. The idea and concept of this work are interesting for presentation in the conference. However, the following comments can be addressed to improve the quality of the work.
strength_weakness: strength_weakness: 1) the title seems unclear, please add some applications of the proposed work in this part to make it more comprehensive.
2) In the abstract section it is mentioned “we focus on developing efficient calibration methods via Bayesian decision-making for the family of compartmental epidemiological models”, the proposed method can be briefly discussed. To make space, the first two sentences of this section can be removed. 
3) Abstract can be supported with some numerical achievements. 
4) More keywords can be added.
5) The working principle of the SIQR compartmental epidemiological model and function network structure of the SIQR model calibration shown in Figs. 1 and 2 can be elaborated in depth.
6) How authors have extracted equations 5, 6, and 7? Please discuss.
7) How authors have obtained the calibration performances plotted in Fig.5?
8) Please explain how authors have achieved the experimental results. 
9) Please add some numerical results to the conclusion.
questions: questions: Please see the comments in the "Weaknesses" box.
title: title: Epidemiological Model Calibration with Bayesian Decision-Making
paper_summary: paper_summary: This study introduces a new method of employing Bayesian optimization (BO) to calibrate compartmental epidemiological models, specifically the SIQR (Susceptible-Infectious-Quarantined-Recovered) model. The authors present a "graybox" BO technique that increases calibration efficiency by utilizing the functional structure of the epidemiological models. When this approach is compared with the conventional "blackbox" BO, it shows improved performance. The theoretical framework and experimental results are robust, showcasing the method's efficiency and potential for broader application. However, the paper primarily uses simulated data, and further validation on real-world data is recommended. Despite this limitation, the methodology is innovative. The findings show that the graybox BO technique can be applied to more complicated models, such as agent-based models, and greatly enhances calibration performance.
strength_weakness: strength_weakness: The suggested strategy is mainly validated by the study using simulated data. While this is helpful, the method's efficacy would be better demonstrated if it were applied to actual data.
The technique relies on specific statistical assumptions (such as Gaussian noise), which could not always hold true in practical situations and could therefore compromise the precision and stability of the model calibration.
Although the graybox BO method is designed to be efficient, the paper does not provide a detailed analysis of its computational complexity compared to other methods.
questions: questions: Can you provide more details on the computational resources required for implementing the graybox BO approach? How does it compare to traditional methods in terms of computational cost?