# Presentation: 333210

## Conducting & Reporting CFD Analyses

## Objective is to achieve a high level of credibility and confidence in the 
results from CFD performed as part of the design and
analysis of a propulsion system.

Credibility and confidence are obtained by demonstrating acceptable 
levels of error and uncertainty as assessed through verification and 
validation.
This discussion is focused on the question...
How good are your CFD results?
Determining this precisely is often difficult, but we can try to understand 
sources of error and ways to demonstrate quality CFD results.

- results from CFD performed as part of the design and
- analysis of a propulsion system.
- Credibility and confidence are obtained by demonstrating acceptable
- levels of error and uncertainty as assessed through *verification* and
*validation*.
- This discussion is focused on the question...
*How good are your CFD results?*
- Determining this precisely is often difficult, but we can try to understand
- sources of error and ways to demonstrate quality CFD results.

- Objective

## The characteristics of flows of interest may include:
Mach number range (Static,Mach 0 to hypersonic, Mach 25)
Shock waves.
High Reynolds numbers (laminar transitioning to turbulent).
Turbulent, viscous flow.
Shock / boundary layer interactions.
Boundary layer separation.
High-temperature effects at hypersonic Mach numbers.
Compressor face effects.
Unsteady flows (hammershocks, unstart)

- Mach number range (Static,Mach 0 to hypersonic, Mach 25)
- Shock waves.
- High Reynolds numbers (laminar transitioning to turbulent).
- Turbulent, viscous flow.
- Shock / boundary layer interactions.
- Boundary layer separation.
- High-temperature effects at hypersonic Mach numbers.
- Compressor face effects.
- Unsteady flows (hammershocks, unstart)

- Flow Characteristics

## The CFD analysis may provide:
Steady-state flow at cruise conditions.
Features of the shock system (positions, cowl stand-off).
Features of the boundary layers (thickness, separation).
Pressure recovery at throat and compressor face.
Spillage (side walls, cowl).
Bleed requirements (slots, bleed holes).
Distortion at compressor face.
Drag.
Off-design flow features.
Unsteady flow features (unstart, stability).
Aerodynamic and thermal loads.

- Steady-state flow at cruise conditions.
- Features of the shock system (positions, cowl stand-off).
- Features of the boundary layers (thickness, separation).
- Pressure recovery at throat and compressor face.
- Spillage (side walls, cowl).
- Bleed requirements (slots, bleed holes).
- Distortion at compressor face.
- Drag.
- Off-design flow features.
- Unsteady flow features (unstart, stability).
- Aerodynamic and thermal loads.

- Objectives of a CFD Analysis

## 1. Provide Qualitative Information.
Provide features of flow field (conceptual, rough estimates of quantities).  Accuracy requirements are low.
2. Provide Incremental Quantities, P.
P = Pbaseline + P
Errors partially cancel for P
P = (P2-P1)actual + (E2-E1) = Pactual + E
Moderate accuracy requirement.
3. Provide Absolute Quantities, P.
Requires high level of accuracy.

- Provide features of flow field (conceptual, rough estimates of quantities).  Accuracy requirements are low.
- 2. Provide Incremental Quantities, P.
- P = Pbaseline + P
- Errors partially cancel for P
- P = (P2-P1)actual + (E2-E1) = Pactual + E
- Moderate accuracy requirement.
- 3. Provide Absolute Quantities, P.
- Requires high level of accuracy.

- Levels of the Use of CFD Results

## Verification
“The process of determining that a model implementation accurately represents the developer’s conceptual description of the model and the solution to the model.” (AIAA-G-077-1998).
Verification checks that there are no programming errors and that 
the coding correctly implements the equations and models. It also 
examines discretization errors in the CFD calculations.
Validation
“The process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model.” (AIAA-G-077-1998).
Validation checks that the CFD results agree with reality as 
observed in experiments, flight tests, or applications.

*“**The process of determining that a model implementation accurately represents the developer’s conceptual description of the model and the solution to the model.” (AIAA-G-077-1998).*
- Verification checks that there are no programming errors and that
- the coding correctly implements the equations and models. It also
- examines discretization errors in the CFD calculations.
- Validation
*“**The process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model.” (AIAA-G-077-1998).*
- Validation checks that the CFD results agree with reality as
- observed in experiments, flight tests, or applications.

- Verification and Validation

## Uncertainty
“A potential deficiency in any phase or activity of the modeling process that is due to the lack of knowledge.” (AIAA-G-077-1998).
The use of potential indicates that deficiencies may or may not exist.
Error
“A recognizable deficiency in any phase or activity and simulation that is not due to lack of knowledge.” (AIAA-G-077-1998).
The deficiencies are knowable up examination. Acknowledged errors have procedures for identifying and removing them.  Unacknowledged errors are undiscovered and no set procedures exist to find them.

*“**A potential deficiency in any phase or activity of the modeling process that is due to the **lack of knowledge**.” (AIAA-G-077-1998).*
- The use of *potential *indicates that deficiencies may or may not exist.
**Error**
*“**A recognizable deficiency in any phase or activity and simulation that is not due to lack of knowledge.” (AIAA-G-077-1998).*
- The deficiencies are knowable up examination. *Acknowledged* errors have procedures for identifying and removing them.  U*nacknowledged* errors are undiscovered and no set procedures exist to find them.

- Uncertainty and Error

## Total Error 	= Physical Modeling Error 
			+ Discretization Error
			+ Programming Error
			+ Computer Round-off Error
			+ Usage Error 
			+ Post-Processing Error
Physical modeling errors are usually acknowledged and may be quantified.  
Discretization errors are grid dependent but a priori knowledge of the proper grid for a desired level of accuracy is generally lacking.
Programming errors should have been minimized by programmers.  
Computer round-off errors are usually negligible or controllable. 
Usage errors are minimized by proper training.
Post-processing errors are usually controllable.

- 			+ **Discretization Error**
- 			+ **Programming Error**
- 			+ **Computer Round-off Error**
- 			+ **Usage Error**
- 			+ **Post-Processing Error**
_*Physical modeling errors*_ are usually acknowledged and may be quantified.
_*Discretization errors*_ are grid dependent but a priori knowledge of the proper grid for a desired level of accuracy is generally lacking.
_*Programming errors*_ should have been minimized by programmers.
_*Computer round-off errors*_ are usually negligible or controllable.
_*Usage errors*_ are minimized by proper training.
_*Post-processing errors*_ are usually controllable.

- Errors in CFD Results

## Many physical models are used in CFD and we must understand the 
limits and possible errors with each model:
Spatial Dimensions. (2D, axisymmetric, 3D)
Temporal Dimensions. (steady-state, unsteady)
Equations. (Navier-Stokes, approximations)
Turbulence Model.
Thermodynamic and Transport Property model.
Chemistry Model. 
Flow Boundary Conditions (inflow, outflow).
Compressor Face Model.
Bleed / Blowing Model.
Vortex Generator Model.

- limits and possible errors with each model:
- Spatial Dimensions. (2D, axisymmetric, 3D)
- Temporal Dimensions. (steady-state, unsteady)
- Equations. (Navier-Stokes, approximations)
- Turbulence Model.
- Thermodynamic and Transport Property model.
- Chemistry Model.
- Flow Boundary Conditions (inflow, outflow).
- Compressor Face Model.
- Bleed / Blowing Model.
- Vortex Generator Model.

- Physical Models in CFD

## Errors that exist because continuum flow equations and 
physical models are represented in a discrete domain of 
space (grid) and time.

Level of error is function of interactions between the solution 
and the grid. Since solution is unknown before-hand, the grid 
generation may not produce the optimum grid.  

Discretization error is quantified through verification 
methods that examine grid convergence.

Discretization error is controllable.

- physical models are represented in a discrete domain of
- space (grid) and time.
- Level of error is function of interactions between the solution
- and the grid. Since solution is unknown before-hand, the grid
- generation may not produce the optimum grid.
- Discretization error is quantified through **verification**
- methods that examine grid convergence.
- Discretization error is controllable.

- Discretization Error

## Convergence used in two ways:
Iterative Convergence
Simulations should demonstrate iterative convergence.  As the algorithm 
iterates the solution, the simulation results approach a fixed value 
(residuals drop and level off).  This applies to both steady and unsteady 
flow solutions.   In the case of unsteady flows, the iterative convergence 
applies to iterations over a time step.

Grid Convergence
As the grid spacing is reduced, the simulation results become insensitive 
to the grid and approach the continuum results.

**Iterative Convergence**
- Simulations should demonstrate iterative convergence.  As the algorithm
- iterates the solution, the simulation results approach a fixed value
- (residuals drop and level off).  This applies to both steady and unsteady
- flow solutions.   In the case of unsteady flows, the iterative convergence
- applies to iterations over a time step.
**Grid Convergence**
- As the grid spacing is reduced, the simulation results become insensitive
- to the grid and approach the *continuum* results.

- Iterative and Grid Convergence

## Iterative Convergence can be demonstrated in several ways:
Value of the largest change in the solution over an iteration, Qbig.
Iteration history of the L2 norm of the equation residual.
Examine conservative variables.
Iteration history of a local or global quantity of the flow.
Usually some measure of all of these are used to determine iterative 
convergence.
The largest change in the solution over an iteration is simply

- Value of the largest change in the solution over an iteration, Qbig.
- Iteration history of the L2 norm of the equation residual.
- Examine conservative variables.
- Iteration history of a local or global quantity of the flow.
- Usually some measure of all of these are used to determine iterative
- convergence.
- The largest change in the solution over an iteration is simply

- Iterative Convergence

## The L2 norm of Q is commonly called the residual of the equation solution 
and is computed as

where 	N, total number of grid points
		M, number of elements in Q (5 for Navier-Stokes Equations)
It represents the change in the solution over an iteration averaged over all the grid points and equations.  
The L2 residual is usually displayed and plotted as its log values, [log(L2)].  
This shows the order-of-magnitude of the change in the solution.
Generally desire the L2 residual to approach zero with iterations.

- and is computed as
- where 	N, total number of grid points
- 		M, number of elements in Q (5 for Navier-Stokes Equations)
- It represents the change in the solution over an iteration averaged over all the grid points and equations.
- The L2 residual is usually displayed and plotted as its log values, [log(L2)].
- This shows the order-of-magnitude of the change in the solution.
- Generally desire the L2 residual to approach zero with iterations.

- L2 Residual

## Residual Plots

- The behavior of the
- residual is not always
- so smooth.
- Unsteadiness (real or
- pseudo) in the
- solution (typical of
- turbulent and
- separated flows) may
- limit how far the
- residual drops.
- Often a couple of
- orders-of-magnitude
- drop is all one
- achieves.

## Residual Plots (continued)

- Spalart-Allmaras turbulence model equations

## The solution of the conservation equations must satisfy those conservative 
principles.  Iterative convergence can be demonstrated through examination of 
the conservative variables:
For example, one can examine the conservation of mass through a duct:

- principles.  Iterative convergence can be demonstrated through examination of
- the conservative variables:
- For example, one can examine the conservation of mass through a duct:

- Examine Conservative Variables

- Variation of the mass flow through the duct.

- Statistical analysis of the variation.  This indicates the level of iterative convergence error in determining the mass flow through the duct.

## The change in a local or global property is perhaps the best measure of iterative 
convergence since it typically directly monitors the quantity that is important for 
the engineering analysis.

- convergence since it typically directly monitors the quantity that is important for
- the engineering analysis.

- Change in Local or Global Quantity

- Lift on the ONERA M6 wing with number of cycles.

- Local quantity is point value.  Global quanity is integrated over a portion of the flow field.
- Examples include:
-   Lift on a wing
-   Drag on a wing
-   Inlet recovery
-   Heat flux
- Global quantities usually show a smooth convergence.

## Verification assessment has two aspects:
Verification of a Code
Verifies that code has no programming errors and that the coding 
correctly implements the equations and models.  The methods 
involve examining modules of code, checking basic assumptions 
(mass conservation), and comparing results to analytic results.
Verification of a Calculation
Verifies that a calculation (simulation) demonstrates a certain level 
of accuracy. The primary method for the verification of a calculation 
is the grid convergence study. 

**Verification of a Code**
- Verifies that code has no programming errors and that the coding
- correctly implements the equations and models.  The methods
- involve examining modules of code, checking basic assumptions
- (mass conservation), and comparing results to analytic results.
**Verification of a Calculation**
- Verifies that a calculation (simulation) demonstrates a certain level
- of accuracy. The primary method for the verification of a calculation
- is the ***grid convergence study***.

- Verification Assessment

## Quantifies:
1. “ordered” discretization error band (related to grid size by order p)
2. p, order of grid convergence (order-of-accuracy)
3. continuum or “zero grid spacing’’ value of observed quantity
Approach:
Assumed or demonstrated that all other error terms are negligible,  minimized, accepted, or under control.
Perform CFD solution on two or more grids of increased refinement.
Solutions must be in the “asymptotic range of convergence’’.
If three solutions, can then compute order of convergence, p.
Use Richardson extrapolation to compute continuum value.
Compute Grid Convergence Indices (GCI) as discretization error band.

- 1. “*ordered*” discretization error band (related to grid size by order ***p***)
- 2. ***p***, order of grid convergence (order-of-accuracy)
- 3. continuum or “zero grid spacing’’ value of observed quantity
**Approach**:
- Assumed or demonstrated that all other error terms are negligible,  minimized, accepted, or under control.
- Perform CFD solution on two or more grids of increased refinement.
- Solutions must be in the “asymptotic range of convergence’’.
- If three solutions, can then compute order of convergence, ***p***.
- Use Richardson extrapolation to compute continuum value.
- Compute Grid Convergence Indices (**GCI**) as discretization error band.

- Grid Convergence Study

## Viscous wall spacing set for y+ < 1.0 for fine grid.
Flow field resolution set to resolve shock system.
Grid density to keep grid spacing ratio below 15%.
Use ICEM CFD to construct and GRIDGEN to improve.
Subset grid to remove some sub-layer grid points for wall function.
Best if grid refinement ratio, r =  h2  / h1 = 2.0 
h is grid spacing.
Minimum, r > 1.1
WIND allows for grid sequencing (r = 2).

- Flow field resolution set to resolve shock system.
- Grid density to keep grid spacing ratio below 15%.
- Use ICEM CFD to construct and GRIDGEN to improve.
- Subset grid to remove some sub-layer grid points for wall function.
- Best if grid refinement ratio, *r =  h**2**  **/ h**1* = 2.0
*h* is grid spacing.
- Minimum, *r* > 1.1
- WIND allows for grid sequencing (*r* = 2).

- Grid Generation Considerations

## Example grid convergence study with three grids. 
Mach 2.35.
r = h2 / h1 = 2.
Asymptotic range observed in plot.
Richardson extrapolation.

- Mach 2.35.
*r = h**2 **/ h**1* = 2.
- Asymptotic range observed in plot.
- Richardson extrapolation.

- Example: Supersonic Diffuser

## Order of grid convergence, p.
Errors reduce as grid is refined.

h is the grid spacing.
C is a constant.
Use three values to solve for order of convergence.

For this example, p = 1.786

- Errors reduce as grid is refined.
*h* is the grid spacing.
*C* is a constant.
- Use three values to solve for order of convergence.
- For this example, p = 1.786

- Order of Convergence

## Extrapolation of two quantities to continuum value at “zero grid spacing”.
Provides estimate of error. 
Extrapolate is of higher order.

For this example,  (p2/p0)h=0 = 0.97130

- Provides estimate of error.
- Extrapolate is of higher order.
- For this example,  (p2/p0)h=0 = 0.97130

- Richardson Extrapolation

## Standardized method for reporting discretization error.
Based on Richardson extrapolation.
Error band on fine grid solution.
Expressed as percentage.

FS is factor of safety (FS = 1.25 if three or more grids)
For this example, 
GCI12 = 0.1031%  (0.0010)
GCI23 = 0.3562%  (0.0035)
“Pressure recovery is 0.971 with grid error of  0.001.”

- Based on Richardson extrapolation.
- Error band on fine grid solution.
- Expressed as percentage.
- FS is factor of safety (FS = 1.25 if three or more grids)
- For this example,
- GCI12 = 0.1031%  (0.0010)
- GCI23 = 0.3562%  (0.0035)
- “Pressure recovery is 0.971 with grid error of  0.001.”

- Grid Convergence Index (CGI)

## Solutions are in range in which errors decrease at rate denoted by order of convergence, p.
Check that constant C is indeed constant over solutions,

Check with GCI values over three solutions,

For this example, ratio = 1.002

- Check that constant *C* is indeed constant over solutions,
- Check with GCI values over three solutions,
- For this example, ratio = 1.002

- Asymptotic Range of Convergence

## Validation assessment addresses if CFD code agrees with reality through comparison to experiments.
Experiment must also address its own errors.
Unit  Benchmark  Subsystem  Complete System.
Set of validation cases are being collected and studied that cover the range of flow features present in inlets (turbulent boundary layers, shocks, shock / boundary layer interactions, subsonic ducts, supersonic diffusers).
NPARC Alliance Validation Archive:
www.grc.nasa.gov/www/wind/valid

- Experiment must also address its own errors.
- Unit  Benchmark  Subsystem  Complete System.
- Set of validation cases are being collected and studied that cover the range of flow features present in inlets (turbulent boundary layers, shocks, shock / boundary layer interactions, subsonic ducts, supersonic diffusers).
- NPARC Alliance Validation Archive:
- www.grc.nasa.gov/www/wind/valid

- Validation Assessment

## In summary, for a CFD analysis, report the following:
Objective of CFD analysis
Geometry simplifications
Grid resolution
Boundary and initial conditions
Equations and physical models
Algorithm settings
Iterative convergence criteria
Grid convergence criteria
Results (values, plots, visualizations)
Errors that can be quantified
Sensitivity to models and parameters (turbulence, chemistry)

- Objective of CFD analysis
- Geometry simplifications
- Grid resolution
- Boundary and initial conditions
- Equations and physical models
- Algorithm settings
- Iterative convergence criteria
- Grid convergence criteria
- Results (values, plots, visualizations)
- Errors that can be quantified
- Sensitivity to models and parameters (turbulence, chemistry)

- Reporting CFD Results

## A sample of books and resources for learning more on CFD:
Anderson, D.A., Tannehill, J.C., and Pletcher, R.H., Computational Fluid Mechanics and Heat Transfer, McGraw-Hill, 1984.
Anderson, J.D., Computational Fluid Dynamics, McGraw-Hill, 1995.
Hirsch, C., Numerical Computation of Internal and External Flows (2 Volumes), John Wiley & Sons, 1988 & 1990.
Hoffmann, K.A. and S.T. Chiang, Computational Fluid Dynamics (3 Volumes), EES Books, 2000.  (also AIAA coarse).
Wilcox, D.C., Turbulence Modeling for CFD, DCW Industries, 1998.
AIAA Courses (CFD, Turbulence)
Grid Generation training (ICEM, Pointwise, VGRID)

- Anderson, D.A., Tannehill, J.C., and Pletcher, R.H., _Computational Fluid Mechanics and Heat Transfer_, McGraw-Hill, 1984.
- Anderson, J.D., Computational Fluid Dynamics, McGraw-Hill, 1995.
- Hirsch, C., _Numerical Computation of Internal and External Flows_ (2 Volumes), John Wiley & Sons, 1988 & 1990.
- Hoffmann, K.A. and S.T. Chiang, _Computational Fluid Dynamics_ (3 Volumes), EES Books, 2000.  (also AIAA coarse).
- Wilcox, D.C., _Turbulence Modeling for CFD_, DCW Industries, 1998.
- AIAA Courses (CFD, Turbulence)
- Grid Generation training (ICEM, Pointwise, VGRID)

- Further Information on CFD