# Multi-Fidelity Gradient-Based Optimization for High-Dimensional Aeroelastic Configurations
## 2 Modeling and Analysis
### Multi-Fidelity, Gradient-Based Design Optimization
Hence, the first iteration costs at least two high-fidelity function gradient evaluations (plus the corrected low-fidelity line search), while the high-fidelity model is evaluated only once per subsequent iteration. Because inaccuracies in the multi-fidelity model may lead the optimizer to attempt a step that increases the high-fidelity objective, an accept-reject decision is adopted from TRMM (part of driver Step 9). If an attempted step increases the objective, the iteration is repeated from the same design point. However, the additional data improve the situation in two ways. First, the data from the failed step are added to the approximate Hessian to improve future steps, as outlined in [14]. Second, the rejected point is likely near the subsequent expected point and can improve the multi-fidelity correction. The MF-BFGS also adopts the notion of a trust region from TRMM. This is applied as box constraints on computing the expected point from the quadratic sub-problem. The trust region is potentially adjusted each iteration using a heuristic based on the ratio \(\rho\) of expected improvement to actual improvement,\[\rho^{(k)}=\frac{\hat{f}_{h}\Big{(}\mathbf{D}^{(k+1)}\Big{)}-f_{h}\Big{(}\mathbf{D} ^{(k)}\Big{)}}{f_{h}\Big{(}\mathbf{D}^{(k+1)}\Big{)}-f_{h}\big{(}\mathbf{D}^{(k) }\big{)}}. \tag{29}\]  
If the accuracy ratio is too small, the trust region size \(\Delta\) is halved. If it is near one, and the new point lies on the trust region boundary; the size is doubled. Otherwise, the trust region size remains the same. \[\Delta^{(k+1)}= \begin{cases}0.5\Delta^{(k)}&\text{if }\rho^{(k)}\leq 0.25\\ \Delta^{(k)}&\text{if }0.25<\rho^{(k)}\leq 0.75\\ \gamma\Delta^{(k)}&\text{if }0.75<\rho^{(k)}\leq 1.25\\ \Delta^{(k)}&\text{if }1.25<\rho^{(k)}\end{cases}  \tag{30}\]  
\[\gamma= \begin{cases}2&\text{if }\|\mathbf{D}^{(k+1)}-\mathbf{D}^{(k)}\|_{ \infty}=\Delta^{(k)}\\ 1&\text{if }\|\mathbf{D}^{(k+1)}-\mathbf{D}^{(k)}\|_{\infty}<\Delta^{(k)}. \end{cases}  \tag{31}\]  
The scaling term \(\gamma\) is based on whether or not the distance from the current point to the next point is equal to the trust region size in any one dimension. These coefficients are selected to be consistent with other examples in the literature (e.g., [16; 17]). Many implementations allow the trust region to grow in size for any ratio greater than one, indicating the objective function reduced by more than anticipated. The contention here, however, is that maintaining the trust region when the ratio is too large emphasizes finding a size where the multi-fidelity correction will be accurate in subsequent iterations. The MF-BFGS implementation was demonstrated on a collection of simple test functions in [14]. Across 105 combinations of test functions, dimensionalities, and starting points, the MF-BFGS required fewer high-fidelity function calls than single-fidelity BFGS in 52% of cases and the same number of evaluations 7% of cases. Compared to TRMM also using the multi-fidelity polynomial chaos, MF-BFGS required fewer or the same number of high-fidelity evaluations in 66% of cases, though MF-BFGS performed more favorably as dimensionality increased up to twenty-five dimensions. Similar performance was also achieved using gradient-enhanced, multi-fidelity kriging rather than polynomial chaos in MF-BFGS, though problems were limited to five dimensions. MF-BFGS was also demonstrated on a simplified, seven-dimensional aero-structural problem in [41]. While multi-fidelity performance relative to high-fidelity optimization varied over the course of optimization, MF-BFGS achieved greater objective reductions at lower cost in early iterations for three of four cases and found superior designs in half of cases given a fixed budget. However, the lack of analytic gradients necessitated finite differencing, and noisy gradients were believed to hinder the convergence of the optimization in both high- and multi-fidelity.