SOME ADVANCED CONCEPTS IN DISCRETE

AERODYNAMIC SENSITIVITY ANALYSIS

Arthur C. Taylor III ^\*^ Lawrence L. Green^†^

##  Perry A. Newman^†^ Michele M. Putko^\*^

### ***Department of Mechanical Engineering*^\*^**

# *Old Dominion University*

#### Norfolk, VA 23529

# *Multidisciplinary Optimization Branch*^†^

#### NASA Langley Research Center

# *Hampton, VA 23681*

# 

# http://fmad-www.larc.nasa.gov/mdob/MDOB 

# THREE KEY IDEAS

# An Alternative Perspective on the Adjoint-Variable Method is Presented.

1.  A Highly-Efficient, Non-Iterative, ADIFOR-Assisted Method for
    Computing Second-Order Sensitivity Derivatives (SO SDs) is Presented
    and Verified.

2.  Significant Improvements in the Computational Efficiency of the
    Reverse-Mode Capability of ADIFOR 3.0 for First-Order Sensitivity
    Derivatives (FO SDs) are Achieved Through
    [Incremental--Iterative]{.underline} Implementations.

# BASIC EQUATIONS

**F** = **F**(**Q**(**b**), **X**(**b**), **b**)

**R** = **R**(**Q**(**b**), **X**(**b**), **b**) = **0**

**F** ≡ Aerodynamic Output Functions (Dimension: NOF)

**b** ≡ Aerodynamic Design Variables (Dimension: NDV)

**Q** ≡ Aerodynamic Field Variables

**X** ≡ Computational Grid

**R** ≡ Steady-State Discrete Nonlinear Flow Equations

# INDICIAL NOTATION

$F_{\text{ij}}^{'} \equiv \frac{\text{dF}_{i}}{\text{db}_{j}} = \left( \frac{dF}{db} \right)_{\text{ij}}$
$R_{\text{lj}}^{'} \equiv \frac{\text{dR}_{l}}{\text{db}_{j}} = \left( \frac{dR}{db} \right)_{\text{lj}}$

$Q_{\text{mj}}^{'} \equiv \frac{\text{dQ}_{m}}{\text{db}_{j}} = \left( \frac{dQ}{db} \right)_{\text{mj}}$
$X_{\text{pj}}^{'} \equiv \frac{\text{dX}_{p}}{\text{db}_{j}} = \left( \frac{dX}{db} \right)_{\text{pj}}$

# FO SDs DIRECT METHOD (FORWARD-MODE)

$$F_{\text{ij}}^{'} = \frac{\text{dF}_{i}}{\text{db}_{j}} = \frac{\partial F_{i}}{\partial Q_{m}}Q_{\text{mj}}^{'} + \frac{\partial F_{i}}{\partial X_{p}}X_{\text{pj}}^{'} + \frac{\partial F_{i}}{\partial b_{j}}$$

$$R_{\text{lj}}^{'} = \frac{\text{dR}_{l}}{\text{db}_{j}} = \frac{\partial R_{l}}{\partial Q_{m}}Q_{\text{mj}}^{'} + \frac{\partial R_{l}}{\partial X_{p}}X_{\text{pj}}^{'} + \frac{\partial R_{l}}{\partial b_{j}} = 0_{\text{lj}}$$

# 

# (NDV solutions for $Q_{\text{mj}}^{'}$)

# 

#  

# FO SDs ADJOINT-VARIABLE METHOD (REVERSE-MODE)

-   Lagrange Multiplier Viewpoint (Conventional)

-   Derivative (Chain Rule) Viewpoint

# ADJOINT-VARIABLE METHOD (CHAIN-RULE PERSPECTIVE)

$$\frac{\partial F_{i}}{\partial R_{l}}\frac{\partial R_{l}}{\partial Q_{m}} = \frac{\partial F_{i}}{\partial Q_{m}}$$

$$\frac{\partial F_{i}}{\partial Q_{m}}Q_{\text{mj}}^{'} = \frac{\partial F_{i}}{\partial R_{l}}\left( \frac{\partial R_{l}}{\partial Q_{m}}Q_{\text{mj}}^{'} \right) = - \frac{\partial F_{i}}{\partial R_{l}}\left( \frac{\partial R_{l}}{\partial X_{p}}X_{\text{pj}}^{'} + \frac{\partial R_{l}}{\partial b_{j}} \right)$$

$$F_{\text{ij}}^{'} = \frac{\text{dF}_{i}}{\text{db}_{j}} = - \frac{\partial F_{i}}{\partial R_{l}}\left( \frac{\partial R_{l}}{\partial X_{p}}X_{\text{pj}}^{'} + \frac{\partial R_{l}}{\partial b_{j}} \right) + \frac{\partial F_{i}}{\partial X_{p}}X_{\text{pj}}^{'} + \frac{\partial F_{i}}{\partial b_{j}}$$

$\left( \text{NOF solutions for}\frac{\partial F_{i}}{\partial R_{l}} \right)$

# THE RELATIONSHIP

$- \frac{\partial F_{i}}{\partial R_{l}} = \lambda_{\text{il}}$

-   The Adjoint -Variables (Lagrange-Multipliers) are
    [Derivatives]{.underline}

-   ADIFOR 3.0 Reverse-Mode computes them as such

# SO SDs

# INDICIAL NOTATION

$$F_{\text{ijk}}^{''} \equiv \frac{d^{2}F_{i}}{\text{db}_{k}\text{db}_{j}} = \left( \frac{d^{2}F}{{db}^{2}} \right)_{\text{ijk}}$$

$$R_{\text{ljk}}^{''} \equiv \frac{d^{2}R_{l}}{\text{db}_{k}\text{db}_{j}} = \left( \frac{d^{2}R}{{db}^{2}} \right)_{\text{ljk}}$$

$Q_{\text{mjk}}^{''} \equiv \frac{d^{2}Q_{m}}{\text{db}_{k}\text{db}_{j}} = \left( \frac{d^{2}Q}{{db}^{2}} \right)_{\text{mjk}}$

$$X_{\text{pjk}}^{''} \equiv \frac{d^{2}X_{p}}{\text{db}_{k}\text{db}_{j}} = \left( \frac{d^{2}X}{{db}^{2}} \right)_{\text{pjk}}$$

# SO SDs DIRECT METHOD (FORWARD-MODE)

$$F_{\text{ijk}}^{''} = \frac{d^{2}F_{i}}{\text{db}_{k}\text{db}_{j}} = \frac{\partial F_{i}}{\partial Q_{m}}Q_{\text{mjk}}^{''} + \frac{\partial F_{i}}{\partial X_{p}}X_{\text{pjk}}^{''} + \frac{DF_{\text{ij}}^{'}}{\text{Db}_{k}}$$

$R_{\text{ljk}}^{''} = \frac{d^{2}R_{l}}{\text{db}_{k}\text{db}_{j}} = \frac{\partial R_{l}}{\partial Q_{m}}Q_{\text{mjk}}^{''} + \frac{\partial R_{l}}{\partial X_{p}}X_{\text{pjk}}^{''} + \frac{DR_{\text{lj}}^{'}}{\text{Db}_{k}} = 0_{\text{ljk}}$

$$\frac{D()}{\text{Db}_{k}} \equiv \frac{\partial()}{\partial Q_{n}}Q_{\text{nk}}^{'} + \frac{\partial()}{\partial X_{q}}X_{\text{qk}}^{'} + \frac{\partial()}{\partial b_{k}}$$

# NDV solutions for $Q_{\text{mj}}^{'}$, followed by

# (NDV^2^+NDV) / 2 solutions for $Q_{\text{mjk}}^{''}$ 

# NOTATIONAL COMPACTNESS

$$\frac{D()}{\text{Db}_{k}} \equiv \frac{\partial()}{\partial Q_{n}}Q_{\text{nk}}^{'} + \frac{\partial()}{\partial X_{q}}X_{\text{qk}}^{'} + \frac{\partial()}{\partial b_{k}}$$

# DERIVATIVE COMPLEXITY

$$\begin{array}{r}
\frac{DF_{\text{ij}}^{'}}{\text{Db}_{k}} = \frac{\partial F_{\text{ij}}^{'}}{\partial Q_{n}}Q_{\text{nk}}^{'} + \frac{\partial F_{\text{ij}}^{'}}{\partial X_{q}}X_{\text{qk}}^{'} + \frac{\partial F_{\text{ij}}^{'}}{\partial b_{k}} \\
\frac{\partial^{2}F_{i}}{\partial Q_{n}\partial Q_{m}}Q_{\text{mj}}^{'}Q_{\text{nk}}^{'} + \frac{\partial^{2}F_{i}}{\partial Q_{n}\partial X_{p}}X_{\text{pj}}^{'}Q_{\text{nk}}^{'} + \frac{\partial^{2}F_{i}}{\partial Q_{n}\partial b_{j}}Q_{\text{nk}}^{'} \\
 + \frac{\partial^{2}F_{i}}{\partial X_{q}\partial Q_{m}}Q_{\text{mj}}^{'}X_{\text{qk}}^{'} + \frac{\partial^{2}F_{i}}{\partial X_{q}\partial X_{p}}X_{\text{pj}}^{'}X_{\text{qk}}^{'} + \frac{\partial^{2}F_{i}}{\partial X_{q}\partial b_{j}}X_{\text{qk}}^{'} \\
 + \frac{\partial^{2}F_{i}}{\partial b_{k}\partial Q_{m}}Q_{\text{mj}}^{'} + \frac{\partial^{2}F_{i}}{\partial b_{k}\partial X_{p}}X_{\text{pj}} + \frac{\partial^{2}F_{i}}{\partial b_{k}\partial b_{j}}
\end{array}$$

# DERIVATIVE COMPLEXITY

$\begin{array}{r}
\frac{DR_{\text{ij}}^{'}}{\text{Db}_{k}} = \frac{\partial R_{\text{ij}}^{'}}{\partial Q_{n}}Q_{\text{nk}}^{'} + \frac{\partial R_{\text{ij}}^{'}}{\partial X_{q}}X_{\text{qk}}^{'} + \frac{\partial R_{\text{ij}}^{'}}{\partial b_{k}} \\
\frac{\partial^{2}R_{i}}{\partial Q_{n}\partial Q_{m}}Q_{\text{mj}}^{'}Q_{\text{nk}}^{'} + \frac{\partial^{2}R_{i}}{\partial Q_{n}\partial X_{p}}X_{\text{pj}}^{'}Q_{\text{nk}}^{'} + \frac{\partial^{2}R_{i}}{\partial Q_{n}\partial b_{j}}Q_{\text{nk}}^{'} \\
 + \frac{\partial^{2}R_{i}}{\partial X_{q}\partial Q_{m}}Q_{\text{mj}}^{'}X_{\text{qk}}^{'} + \frac{\partial^{2}R_{i}}{\partial X_{q}\partial X_{p}}X_{\text{pj}}^{'}X_{\text{qk}}^{'} + \frac{\partial^{2}R_{i}}{\partial X_{q}\partial b_{j}}X_{\text{qk}}^{'} \\
 + \frac{\partial^{2}R_{i}}{\partial b_{k}\partial Q_{m}}Q_{\text{mj}}^{'} + \frac{\partial^{2}R_{i}}{\partial b_{k}\partial X_{p}}X_{\text{pj}} + \frac{\partial^{2}R_{i}}{\partial b_{k}\partial b_{j}}
\end{array}$

# EFFICIENT, NON-ITERATIVE METHOD

$$F_{\text{ijk}}^{''} = \left( \frac{\partial F_{i}}{\partial X_{p}} + \lambda_{\text{il}}\frac{\partial R_{l}}{\partial X_{p}} \right)X_{\text{pjk}}^{''} + \frac{DF_{\text{ij}}^{'}}{\text{Db}_{k}} + \lambda_{\text{il}}\frac{DR_{\text{lj}}^{'}}{\text{Db}_{k}}$$

-   NDV solutions for $Q_{\text{mj}}^{'}$

```{=html}
<!-- -->
```
-   NOF solutions for $\lambda_{\text{il}}$

```{=html}
<!-- -->
```
-   Preferred if (NDV^2^+NDV)/2 \> NOF

# AIRFOIL EXAMPLE PROBLEM

-   NACA 0012

-   Euler Equations

-   *α* = 1.0°

-   *M*~∞~ = 0.80

-   Transonic flow

-   Computational Grid, 129 x 33 points

-   Upwind, Cell-Centered Finite-Volume Solver

(Continued \>\>\>)

**AIRFOIL EXAMPLE PROBLEM**

-   3 Design Variables

> g (geometric shape)
>
> α (angle-of-attack)
>
> M~∞~ (Freestream Mach. Number)

-   3 Output Functions

> C~L~ (Lift Coefficient)
>
> C~D~ (Drag Coefficient)
>
> C~M~ (Pitching-Moment Coefficient)

# FO SDs

# (Forward-Mode & Reverse-Mode)

  ----------------------------------------- ------- ---------------------- -----------------------------
                                            b~j~    Forward-Mode           Reverse-Mode

  $$\frac{\text{dC}_{L}}{\text{db}_{j}}$$   g       +0.1405406E+00         +0.1405406E+00

                                            α       -0.1087323E-01         -0.1087323E-01

                                            M~∞~    -0.4672729E-01         -0.4672729E-01

  $$\frac{\text{dC}_{D}}{\text{db}_{j}}$$   g       +0.1761807E+02         +0.1761807E+02

                                            α       +0.1158625E+01         +0.1158625E+01

                                            M~∞~    -0.2382580E+01         -0.2382580E+01

  $$\frac{\text{dC}_{M}}{\text{db}_{j}}$$   g       +0.3171492E+01         +0.3171492E+01

                                            α       +0.5955598E+01         +0.5955598E+01

                                            M~∞~    -0.1603002E+01         -0.1603002E+01
  ----------------------------------------- ------- ---------------------- -----------------------------

-   FO SDs from Hand-Differentiated Coding

-   Results verified by Finite-Differences; Excellent Agreement

# SO SDs

# NON-ITERATIVE METHOD

  --------------------------------------------------- ----------- ------------------ ----------------- ------------------
                                                      b~j~/b~k~   g                  α                 M~∞~

  $$\frac{d^{2}C_{L}}{\text{db}_{j}\text{db}_{k}}$$   g           +0.248807E+03      +0.250402E+03     +0.205825E+03

                                                      α           +0.250402E+03      +0.184277E+05     +0.160858E+05

                                                      M~∞~        +0.205825E+03      +0.160858E+05     +0.133087E+05

  $$\frac{d^{2}C_{D}}{\text{db}_{j}\text{db}_{k}}$$   g           +0.71777E+02       +0.134379E+02     +0.101304E+02

                                                      α           +0.134379E+02      +0.959310E+03     +0.804021E+03

                                                      M~∞~        +0.101304E+02      +0.804021E+03     +0.662088E+03

  $$\frac{d^{2}C_{M}}{\text{db}_{j}\text{db}_{k}}$$   g           -0.663590E+02      -0.602441E+02     -0.490399E+02

                                                      α           -0.6024415+02      -0.449198E+04     -0.386943+04

                                                      M~∞~        -0.490399E+02      -0.386943E+04     -0.320512E+04
  --------------------------------------------------- ----------- ------------------ ----------------- ------------------

-   Results verified by Finite-Differences of FO SDs; Excellent
    Agreement

# Relative CPU Timings

# SO SDs, Non-Iterative Method

  ----------------------------------- -----------------------------------
  Computational Procedure             \% of Total

  Nonlinear Flow                      5.4

  Forward-Mode FO SDs                 25.5

  Reverse-Mode FO SDs                 69.0

  SO SDs                              **0.1**

  Total                               100.0
  ----------------------------------- -----------------------------------

# Intermediate Summary

# Non-Iterative SO SD Method

-   First, one requires solution of [both]{.underline} methods for FO
    SDs

\- NDV forward-mode solutions for $Q^{'}$

-   NOF reverse-mode solutions for ***[λ]{.underline}***

```{=html}
<!-- -->
```
-   Selective forward-mode application of ADIFOR to strategic pieces of
    the same code used for the FO SDs for accurate construction of
    complicated SO derivative terms

```{=html}
<!-- -->
```
-   Thereafter, NOF x NDV^2^ "[exact]{.underline}" SO SDs (i.e. the
    complete Hessian matrices) calculated essentially "for free"

# FO SDs REVERSE-MODE APPLICATIONS OF ADIFOR 

# TABLE OF ACRONYMS

-   **HD** -- Hand Differentiated / Hand Coded

-   **AD** -- Automatic Differentiation (ADIFOR)

-   **II** -- Incremental-Iterative Solution Algorithm

-   **BB** -- Black-Box Application of AD

-   **AV** -- Adjoint-Variable (Reverse-Mode) Differentiation

#  (Continued \>\>\>\>) 

#  FO SDs TABLE OF ACRONYMS

  -------------------------------------- --------------------------------
  Forward-Mode Method                    Reverse-Mode Method

  HDII\*                                 HDII-AV\*

  ADBB\*                                 ADBB-AV\*

  ADII\*                                 ADII-AV^†^

  \*Previous Studies                     

  ^†^ Present Study                      
  -------------------------------------- --------------------------------

**INCREMENTAL-ITERATIVE APPLICATIONS OF ADIFOR FOR CFD CODES**

Computational Issues Associated With Present Reverse-Mode

Applications are Completely Analogous to Those Seen Previously

With Forward-Mode Applications

**CRITICAL COMPUTATIONAL IDEA**

[Forward Mode]{.underline}**:** ADIFOR computes the
[post-multiplication]{.underline} of a large Jacobian Matrix times a
vector very efficiently; in particular:

$$\frac{\partial R_{l}}{\partial Q_{m}}Q_{\text{mj}}^{'}$$

[Reverse Mode]{.underline}**:** ADIFOR computes the
[pre-multiplication]{.underline} of a large Jacobian Matrix times a
vector very efficiently; in particular:

$$\frac{\partial F_{i}}{\partial R_{l}}\frac{\partial R_{l}}{\partial Q_{m}}$$

# INCREMENTAL-ITERATIVE

# SOLUTION METHODS

Non-linear Flow

$$Q_{m}^{N + 1} = Q_{m}^{N} - P_{\text{ml}}^{N}R_{l}^{N}$$

$$P_{\text{ml}}^{N} = \frac{\partial Q_{m}}{\partial{\overset{\sim}{R}}_{l}^{N}} = \left( \frac{\partial{\overset{\sim}{R}}_{l}^{N}}{\partial Q_{m}} \right)^{- 1}$$

(N = 1, 2, 3, ...)

**FO SDs Forward-Mode**

Incremental--Iterative, Forward-Mode (HDII & ADII)

$$Q_{\text{mj}}^{} = Q_{\text{mj}}^{} - P_{\text{ml}}R_{\text{lj}^{M}}^{'}$$

$$R_{\text{lj}^{M}}^{'} = \frac{\partial R_{l}}{\partial Q_{m}}Q_{\text{mj}}^{} + \frac{\partial R_{l}}{\partial X_{p}}X_{\text{pj}}^{'} + \frac{\partial R_{l}}{\partial b_{j}}$$

(M = 1, 2, 3,...)

Black-Box AD, Forward-Mode (ADBB)

$$Q_{\text{mj}}^{} = Q_{\text{mj}}^{} - P_{\text{ml}}^{N}R_{\text{lj}^{N}}^{'} - P_{\text{mlj}}^{}R_{l}^{N}$$

(N = 1, 2, 3,...)

# FO SDs Reverse-Mode 

Incremental Iterative, Reverse-Mode (HDII-AV & ADII-AV)

$\lambda_{\text{il}}^{M + 1} = \lambda_{\text{il}}^{M} - P_{\text{ml}}G_{\text{im}}^{M}$

$$G_{\text{im}}^{M} \equiv \lambda_{\text{il}}^{M}\frac{\partial R_{l}}{\partial Q_{m}} + \frac{\partial F_{i}}{\partial Q_{m}}$$

# (M = 1, 2, 3,...) 

# RELATIVE CPU TIMING COMPARISON

+-------------------------------------+--------------------------------+
| Reverse-Mode Method Tested          | Relative Timing\*              |
+-------------------------------------+--------------------------------+
| HDII-AV                             | 1.0                            |
+-------------------------------------+--------------------------------+
| ADBB-AV                             | 7.9                            |
+-------------------------------------+--------------------------------+
| ADII-AV                             | 4.7                            |
+-------------------------------------+--------------------------------+
| ADII-AV (with                       | 3.3\*\*                        |
|                                     |                                |
| Single, Frozen Forward Pass)        |                                |
+-------------------------------------+--------------------------------+
| \*CPU                               |                                |
| Time/                               |                                |
| iteration/grid-point/differentiated |                                |
| output-function                     |                                |
+-------------------------------------+--------------------------------+
| \*\*Work-In-Progress, Projected     |                                |
| Timing                              |                                |
+-------------------------------------+--------------------------------+

-   Results for Coarse Grid, 2D Internal Flow Example

-   CFD Code Same as Previous Airfoil Example Problem

Accuracy: "Exact" Agreement among all methods tested, as expected;
verified by Finite-Differences

# COMPUTER MEMORY 

**[Disk Storage (ADIFOR Log Files)]{.underline}**

**ADBB-AV:** Iterative Accumulation of Log Files Unmanageably Large

**ADII-AV**: Log Files do [not]{.underline} accumulate; in contrast,
they are fixed on the first iteration and reused on all subsequent
iterations

# 

# [RAM MEMORY]{.underline}

#  Memory Increase -- Original Flow Code x NOF Computed Concurrently

# CONCLUSIONS (THREE KEY IDEAS)

# 

# An Alternative Perspective on the Adjoint-Variable Method is Presented.

1.  A Highly-Efficient, Non-Iterative, ADIFOR-Assisted Method for
    Computing Second-Order Sensitivity Derivatives (SO SDs) is Presented
    and Verified.

2.  Significant Improvements in the Computational Efficiency of the
    Reverse-Mode Capability of ADIFOR 3.0 for First-Order Sensitivity
    Derivatives (FO SDs) are Achieved Through
    [Incremental--Iterative]{.underline} Implementations.