James R. DeBonis

Glenn Research Center, Cleveland, Ohio

William L. Oberkampf

WLO Consulting, Austin, Texas

Richard T. Wolf, Paul D. Orkwis, and Mark G. Turner

University of Cincinnati, Cincinnati, Ohio

Holger Babinsky

University of Cambridge, Cambridge, United Kingdom

Assessment of Computational Fluid Dynamics 

(CFD) Models for Shock Boundary-Layer

Interaction

NASA/TM—2011-216832

March 2011

AIAA–2010–4823
NASA STI Program . . . in Profi le

Since its founding, NASA has been dedicated to the 

advancement of aeronautics and space science. The 

NASA Scientifi c and Technical Information (STI) 

program plays a key part in helping NASA maintain 

this important role.

The NASA STI Program operates under the auspices 

of the Agency Chief Information Offi cer. It collects, 

organizes, provides for archiving, and disseminates 

NASA’s STI. The NASA STI program provides access 

to the NASA Aeronautics and Space Database and 

its public interface, the NASA Technical Reports 

Server, thus providing one of the largest collections 

of aeronautical and space science STI in the world. 

Results are published in both non-NASA channels 

and by NASA in the NASA STI Report Series, which 

includes the following report types:

• TECHNICAL PUBLICATION. Reports of 

completed research or a major signifi cant phase 

of research that present the results of NASA 

programs and include extensive data or theoretical 

analysis. Includes compilations of signifi cant 

scientifi c and technical data and information 

deemed to be of continuing reference value. 

NASA counterpart of peer-reviewed formal 

professional papers but has less stringent 

limitations on manuscript length and extent of 

graphic presentations.

• TECHNICAL MEMORANDUM. Scientifi c 

and technical fi ndings that are preliminary or 

of specialized interest, e.g., quick release 

reports, working papers, and bibliographies that 

contain minimal annotation. Does not contain 

extensive analysis.

• CONTRACTOR REPORT. Scientifi c and 

technical fi ndings by NASA-sponsored 

contractors and grantees.

• CONFERENCE PUBLICATION. Collected 

papers from scientifi c and technical 

conferences, symposia, seminars, or other 

meetings sponsored or cosponsored by NASA.

• SPECIAL PUBLICATION. Scientifi c, 

technical, or historical information from 

NASA programs, projects, and missions, often 

concerned with subjects having substantial 

public interest.

• TECHNICAL TRANSLATION. Englishlanguage translations of foreign scientifi c and 

technical material pertinent to NASA’s mission.

Specialized services also include creating custom 

thesauri, building customized databases, organizing 

and publishing research results.

For more information about the NASA STI 

program, see the following:

• Access the NASA STI program home page at 

http://www.sti.nasa.gov

• E-mail your question via the Internet to help@

sti.nasa.gov

• Fax your question to the NASA STI Help Desk 

at 443–757–5803

• Telephone the NASA STI Help Desk at

 443–757–5802

• Write to:

 NASA Center for AeroSpace Information (CASI)

 7115 Standard Drive

 Hanover, MD 21076–1320
James R. DeBonis

Glenn Research Center, Cleveland, Ohio

William L. Oberkampf

WLO Consulting, Austin, Texas

Richard T. Wolf, Paul D. Orkwis, and Mark G. Turner

University of Cincinnati, Cincinnati, Ohio

Holger Babinsky

University of Cambridge, Cambridge, United Kingdom

Assessment of Computational Fluid Dynamics 

(CFD) Models for Shock Boundary-Layer

Interaction

NASA/TM—2011-216832

March 2011

AIAA–2010–4823

National Aeronautics and

Space Administration

Glenn Research Center

Cleveland, Ohio 44135

Prepared for the

28th Applied Aerodynamics Conference

sponsored by the American Institute of Aeronautics and Astronautics

Chicago, Illinois, June 28 to July 1, 2010
Acknowledgments

The authors would like to thank all of those who contributed solutions their solutions for this study. The signifi cant time and 

effort that is represented by each solution is greatly appreciated. The authors would also like to thank the experimentalists

who provided their data for the effort. In particular, Ethan Eagle and Evan Bonny provided valuable insight that assisted

in the uncertainty analysis of their experiments at the University of Michigan.

Available from

NASA Center for Aerospace Information

7115 Standard Drive

Hanover, MD 21076–1320

National Technical Information Service

5301 Shawnee Road

Alexandria, VA 22312

Available electronically at http://www.sti.nasa.gov

Level of Review: This material has been technically reviewed by technical management. 

This report is a formal draft or working 

paper, intended to solicit comments and 

ideas from a technical peer group.

This report contains preliminary fi ndings, 

subject to revision as analysis proceeds.
A workshop on the computational fluid dynamics (CFD) prediction of shock boundarylayer interactions (SBLIs) was held at the 48th AIAA Aerospace Sciences Meeting. As part

of the workshop numerous CFD analysts submitted solutions to four experimentally measured SBLIs. This paper describes the assessment of the CFD predictions. The assessment

includes an uncertainty analysis of the experimental data, the definition of an error metric

and the application of that metric to the CFD solutions. The CFD solutions provided very

similar levels of error and in general it was difficult to discern clear trends in the data. For

the Reynolds Averaged Navier-Stokes methods the choice of turbulence model appeared to

be the largest factor in solution accuracy. Large-eddy simulation methods produced error

levels similar to RANS methods but provided superior predictions of normal stresses.

Nomenclature

u, v, w streamwise, transverse and spanwise velocity components

x, y, z cartesian coordinates

E(f) error metric of f

M∞ freestream Mach number

p0 stagnation pressure

T0 stagnation temperature

Reθ Reynolds number based on momentum thickness

U∞ freestream velocity

< u02 > streamwise normal stress

< v02 > transverse normal stress

< u0v

0 > shear stress

δ boundary-layer thickness

θ shock deflection angle

∆x, ∆y, ∆z spacing of PIV measurement grid

subscripts

cf d computational fluid dynamics obtained quantity

exp experimentally obtained quantity

Abstract 

Assessment of Computational Fluid Dynamics (CFD) Models for 

Shock Boundary-Layer Interaction 

James R. DeBonis 

National Aeronautics and Space Administration 

Glenn Research Center 

Cleveland, Ohio 44135 

William L. Oberkampf 

WLO Consulting 

Austin, Texas 78633 

Richard T. Wolf, Paul D. Orkwis, and Mark G. Turner 

University of Cincinnati 

Cincinnati, Ohio 45221 

Holger Babinsky 

University of Cambridge 

Cambridge, United Kingdom, CB2 1PZ 

NASA/TM—2011-216832 1
I. Introduction

A shock boundary-layer interaction (SBLI) is an important and complex fluid dynamic phenomena that

occurs in supersonic flows. It is one of the primary flow features in supersonic inlets, and if not controlled,

can have a major impact on inlet performace. Predicting SBLIs and understanding how to control them is

a critical technology area for development of propulsion systems for supersonic vehicles. The United States

Air Force, the National Aeronautics and Space Administration (NASA) and numerous aerospace companies

are developing and researching supersonic vehicles.

The SBLI considered in this study consists of an oblique shock wave, impinging on a supersonic flow over

a wall which has formed a turbulent boundary layer. The shock wave imposes an abrupt adverse pressure

gradient. This pressure gradient negatively affects the boundary layer, thickening the layer and perhaps

separating the boundary layer from the wall. The interaction increases turbulence levels in the boundary

layer and typically induces unsteadiness in the shock wave. In wind tunnels and inlets with rectangular

cross-sections the shock also affects the sidewall boundary layer and corner flows resulting in extremely

complex three-dimensional flowfields. An excellent description of the phenomena is given by Delery and

Marvin.1In a supersonic inlet the SBLI creates total pressure loss, flow distortion and in the case of mixedand internal-compression inlets the increased potential of inlet unstart (reduced stability).

The phenomena is complex and difficult to predict using computational fluid dynamics (CFD). Traditionally Reynolds Averaged Navier-Stokes (RANS) CFD methods have been used for SBLI predictions.

RANS methods rely on models to simulate the effect of the turbulence on the mean flow. These models are

developed and adapted for well-behaved boundary layers and shear layers and can not accurately model the

complex flow in a SBLI. Large-eddy simulation (LES), a technique which directly computes the large-scale

turbulence, offers the promise for improved predictions but is just now becoming a practical option for these

types of flows due to increased computing power.

A workshop on the CFD prediction of shock boundary-layer interactions was organized by the American

Institute of Aeronautics and Astronautics (AIAA).2–4 The workshop was held at the 48th AIAA Aerospace

Sciences Meeting in Orlando, Florida. The purpose of the workshop was to share prediction methodologies,

assess the state of the art in SBLI prediction and determine the most promising methods. Four test cases

with extensive experimental data were identified and the geometry and flow conditions for each were provided

to the analysts. The experimental data was also provided for two cases - the cases with the weakest shock

interactions. The data was held back for the two cases with the strongest shock interaction, to assess the

solutions in a true predictive situation. A key component of this effort was to provide an objective assessment

of the error for each of the predictions. This paper discusses the assessment of the CFD predictions for the four

SBLI workshop cases. The assessment includes an evaluation of the experimental uncertainty, a definition

of an error metric, the results of the error assessment for each test case and a discussion of some trends that

were identified in the data.

II. Experimental Data

The data used in the workshop came from two sources. The general setup of the experiment was similar

for both (Fig. 1). A shock generator plate was placed in the test section of a supersonic wind tunnel at an

angle to the freestream flow. This angle, θ, is the shock deflection angle, which determines the strength of

the oblique shock wave. A turbulent boundary layer is present on the floor of the wind tunnel. An oblique

shock wave is formed by the plate and intersects the boundary layer. Particle Image Velocimetry (PIV)

measurements of velocities and turbulent quantities were performed in the region of the shock boundarylayer interaction. The use of PIV, a relatively new technique, for comparison to CFD represents an advance

in the assessment of CFD for SBLIs. Typically wall pressure and skin friction data has been used. PIV

provides a much more detailed look at the flow. Table 1 summarizes the experimental cases.

The geometries for both test setups were provided in IGES format. The basic layouts of the tunnels

and shock generator plates, based on the IGES files, are shown in Fig. 2. A comparison of the wind tunnel

cross-sections including the shock generator plate and boundary-thickness are given in Fig. 3.

NASA/TM—2011-216832 2
θ 

M∞ 

PIV Plane 

y 

x 

Figure 1. Schematic of a SBLI in a supersonic wind tunnel

Organization M∞ θ (deg.) T0 (K) p0 (kPa) Reθ

IUSTI Marseille (UFAST) 2.25 8.0 293 50.5 6,900 open

University of Michigan (CCAS) 2.75 7.75 293 101 6,600 open

University of Michigan (CCAS) 2.75 10.0 293 101 6,600 blind

University of Michigan (CCAS) 2.75 12.0 293 101 6,600 blind

Table 1. Summary of the experimental cases

(a) IUSTI (b) UMich

Figure 2. Wind tunnel and shock generator layout (courtesy of Vishal Bhagwandin, Army Research Laboratory)

NASA/TM—2011-216832 3
IUSTI, Marseille (UFAST) University of Michigan (CCAS)

170 mm

120 mm

! " 10 mm

57 mm

70 mm

! " 10 mm

32 mm

(a) IUSTI

IUSTI, Marseille (UFAST) University of Michigan (CCAS)

170 mm

120 mm

! " 10 mm

57 mm

70 mm

! " 10 mm

32 mm

(b) UMich

Figure 3. Comparison of wind tunnel cross-sections

The two experimental datasets are summarized below. Details of the PIV measurements and their effect

on the experimental uncertainty are given in the following section (Sec. III).

A. IUSTI Mach 2.25 – 8.0 degree case

The first dataset was taken at the Institut Universitaire des Syst`emes Thermiques Industriels (IUSTI) in

Marseille, France.5 This data was also used as the test case for the European Union sponsored research

effort on SBLIs named UFAST.6 The freestream Mach number, M∞, was 2.25 and the deflection angle was

8.0 degrees. The wind tunnel nozzle has converging/diverging walls on both the upper and lower surfaces

and straight sidewalls (Fig. 2(a)). The wind tunnel test section was 120 mm high and 170 mm wide. The

shock generator plate spanned the width of the tunnel, with only a small gap between the plate and tunnel

sidewalls. The data taken included 2D PIV, stereoscopic PIV, laser doppler velocimetry (LDV) and wall

pressures. The data used here for comparison is the PIV data on a streamwise plane (a plane parallel to

the freestream flow and the tunnel sidewall) in the center of the tunnel. The PIV plane was limited to only

the interaction region. The LDV measurements were used to verify the PIV data. The data provided for

comparison included: streamwise and transverse velocities, u and v, normal stresses, √

< u02 > and √< v02 >

and shear stress, − < u0v

0 >.

B. UMich. Mach 2.75 – 7.75, 10.0 and 12.0 degree cases

The remaining datasets were taken at the University of Michigan (UMich.)7 The UMich effort is part of

the U.S. Air Force sponsored Collaborative Center for the Aeronautical Sciences (CCAS). The freestream

Mach number, M∞, was 2.75. The shock generator plate was set at three different angles, 7.75, 10.0 and

12.0 degrees, increasing the strength of the shock and the size of the interaction. The upper wall of the wind

tunnel served as the converging diverging nozzle (Fig. 2(b)). The wind tunnel test section was 70 mm high

and 57 mm wide. The shock generator plate was 32 mm wide and spanned the center portion of the tunnel

(Fig. 3(b)). The narrower plate is an attempt to avoid strong shock boundary-layer interactions on the

tunnel sidewalls. The test section is narrower than the IUSTI test section and the boundary layer and shock

generator create a larger percentage of blockage for the tunnel flow. One would expect significant influence

of the corner flows on the tunnel centerline and a more three-dimensional flowfield. The data consists of

stereo PIV measurements on a streamwise plane in the center of the tunnel and several spanwise planes (Fig.

4). The PIV planes include the interaction region and a portion of the freestream flow above. In addition,

a streamwise data plane was provided to characterize the boundary layer upstream of the interaction. The

data provided included: streamwise, transverse and spanwise velocities, u, v and w, normal stresses, < u02 >,

< v02 > and < w02 >, shear stresses, < u0v

0 >, < u0w0 >, and < v0w0 > and velocity gradients. Note that

there is a difference between the two experiments in the form of the turbulent stresses.

NASA/TM—2011-216832 4
X

Y

Z

Figure 4. Streamwise and spanwise planes for PIV measurements for the UMich. data

III. Experimental Uncertainty Estimation

This section discusses how the uncertainties in the experimental measurements from IUSTI and UMich.

were estimated. For the IUSTI experiment, an estimate of the experimental uncertainty was obtained from

Dussage et al.8 and for the and for the UMich. experiment, an estimate of uncertainty was obtained from

Lapsa and Dahm.7 We also applied an independent method to estimate experimental uncertainty developed

by Oberkampf and coworkers9–11 to the UMich. Data. The method is based entirely on statistically analyzing

the differences in multiple measurements of the various quantities of interest, e.g., u, v, w, as well as normal

stresses and shear stresses.

A. IUSTI Data Uncertainty

Spatial resolution of the stereoscopic PIV technique is limited by the thickness of the laser lightsheet (1

mm) and the size of the analysis windows. The final effective cell size is 16 (horizontal) x 8 (vertical) pixels,

leading to the data being averaged over a volume of 1 mm x 0.5 mm, and a light sheet thickness of 1

mm. No detailed analysis of the uncertainty in the velocity measurements was provided, but the estimated

uncertainty is quoted as 6 m/s.5 This value of uncertainty was used in the comparisons of CFD results with

IUSTI measurements.

An independent assessment of the accuracy of the PIV measurements was obtained by a number of

comparisons with LDV measurements for the same conditions in the wind tunnel. It was found that with

the initial field of view for the PIV that large errors occurred in measured velocities. After the field of view

was significantly reduced, they found good agreement between PIV and LDV measurements for both mean

velocities as well as turbulent stresses.

B. UMich. Data Uncertainty

The parameters for the PIV set-up at the University of Michigan were similar to those used in the IUSTI

experiments, except that data were obtained by stereoscopic PIV along a streamwise plane as well as seven

spanwise planes. The streamwise field of view covers a significantly larger region than each spanwise slice

and for this reason the PIV window size and operating parameters differ. In the streamwise plane a window

of 32x32 pixels (the smallest used in the analysis) was approximately equivalent to 1 mm x 1 mm window

size and the light sheet thickness was 0.5mm. A delay of about 1 μs was set between the two laser pulses.

Spanwise planes use a smaller field of view to obtain better spatial resolution and as a result window sizes

were 0.5 mm x 0.5 mm and the time increment between two laser pulses was around 500 ns.

NASA/TM—2011-216832 5
No comprehensive uncertainty analysis was provided for the measurements, nor were the PIV measurements compared with an independent technique. However, the repeatability of the measurements was

assessed by comparing the results of different test campaigns performed several weeks apart. The different

test campaigns involved completely independent forming and positioning of the laser sheets, positioning of

the stereo cameras, calibration of the camera fields-of-view, and all other aspects associated with set-up and

calibration of the experiment. Comparison of the two results through the boundary layer showed that the

mean u component of velocity was indiscernible between the two campaigns, except for the first three points

near the wall. Comparisons for the mean v component of velocity showed a maximum difference of 9 m/s

and the average difference through the boundary layer of 3 m/s.

One particular issue of concern emerged in recent months due to the work of Ethan Eagle and Evan

Bonny of the Dept. of Aerospace Engineering, University of Michigan, working with the same stero-PIV

instrumentation. The spanwise high-resolution measurements in particular required very short time intervals

between the two laser pulses (500 nm) which are at the limit of what is possible with today’s technology. This

was recognized by the original researchers who measured the actual time delay between two laser pulses at

the start of the investigation and reported that this was somewhat larger than commanded by the software.

However, subsequent research by Eagle and Bonny found that the hardware used for the experiments was

less reliable than previously thought and that the actual delay fluctuates more significantly, giving a velocity

bias error that is directly proportional to the timing discrepancy. It was found that there is on average an

additional time delay of up to 50 ns and a shot-to-shot variation of the order of 10-15 ns. The original data

was obtained with only a single measurement of this timing delay and it is therefore likely that it suffers

from both a systematic bias error and a random fluctuating error. For the data taken along the spanwise

planes these errors might be as large as 10-13%, while the streamwise plane data (which was taken with

larger interval times) may be affected by about half of these estimates. For this reason only the streamwise

planes are used for comparison in this paper. This topic will be revisited in the next section.

C. Statistical Estimation Method

A completely different approach for estimating the experimental uncertainty of measurements obtained in

wind tunnels has been developed by Oberkampf and coworkers.9–11 This technique is a statistical approach

that is based on analyzing comparisons of multiple measurements of quantities of interest. It is usually

referred to as statistical design of experiments (DOE) because it analyzes the final measured result of the

quantities of interest, based on specially designed sampling techniques.12, 13 We were able to apply this

technique to the published UMich. data and take advantage of the fact that the sampling is derived from

two different planes. Here we will only give a brief description of the approach, the key computational steps,

and the results of the analysis.

The approach compares measurements of each of the quantities of interest that are obtained from both the

streamwise and spanwise planes. The quantities of interest can be either directly measured quantities, i.e.,

mean velocity components u, v, w, and u

0

, v0, w0, or they can be quantities derived from the measurements,

such as gradients of these quantities or Reynolds stresses. At each of the seven intersections of the spanwise

planes with the streamwise plane, and for each of the three shock generator angles, the uncertainty in the

quantities of interest can be estimated.

This statistical approach could also be applied to comparisons between multiple measurements of the same

quantity in the same plane. This calculation would independently estimate the repeatability uncertainty that

was discussed above (Sec. B). It is well known that this type of repeatability uncertainty, also referred to as

0th order replication,14 is smaller, sometimes much smaller, than the total uncertainty due to other sources

in an experiment. Although this calculation was not done for the UMich. experiment, it is planned for future

work because the repeatability data has been archived.

When comparisons are made between multiple measurements of the same quantity in different planes,

then in addition to the repeatability uncertainty just mentioned, one can capture a wider range of systematic

(bias) uncertainties. Examples of these systematic uncertainties are any changes in the calibration procedure

of the PIV system (such as spatial positioning of the laser sheet or measurement volume), changes in the

optical setup of the laser sheet used in the PIV system, and changes in PIV window size or spatial resolution.

The effect of these types of systematic uncertainties on the quantities of interest can be quantified by using

streamwise and spanwise measurements because now we have sampled correlated bias uncertainties that

arise from different populations. That is, anytime in an experiment that the quantities of interest can be

measured where some source of uncertainty can be altered, then we are sampling from a modified population.

NASA/TM—2011-216832 6
The more independent each modified population is, the more accurate we are in estimating the true (total)

uncertainty. For example, more independence of the sources of correlated bias uncertainties can be obtained

if one were to measure the velocity components with an LDV technique, or make measurements using a hot

wire technique in a separate wind tunnel facility.

To make comparisons of quantities of interest at the intersection of two planes, one must interpolate both

planes of data to common points in space so that a proper assessment of uncertainty is made. We used

bi-linear interpolation in each plane of data to obtain each quantity of interest at the following intersection

points; x is defined by the x value of the spanwise plane of data, y is defined by the y value of the streamwise

plane of points, and z is defined by z = 0 (the z coordinate the streamwise plane. The spacing of the

measurement points in the streamwise plane was ∆x = ∆y = 0.5024 mm and in the spanwise plane it was

∆y = ∆z = 0.2879 mm. As a result, the bi-linear interpolation of the measurements required no more than

half of each of these distances for the interpolation.

Figure 5 shows the residuals of the u, v, and w mean velocity components for the 21 streamwise and

spanwise plane intersections (seven spanwise plane intersections for each of the three shock generator angles).

Since each intersection has 30 - 35 measurement points in the streamwise plane, this yields a total 681

residuals for each quantity of interest. The residual in Fig. 5 is the difference between the measurement of

the quantity in each plane, and the abscissa is the average value of the measured quantity in both planes.

None of the scatter plots show any clear trend with regard to how the residual changes as a function of

magnitude of velocity, or how it varies with the deflection angle. The v and w plots show a slight trend of

decreasing residuals as a function of magnitude of velocity. It is believed that the random and bias error

in the shot-to-shot timing of the two laser pulses mentioned above is responsible for the large scatter in the

residuals, particularly for the low magnitudes of velocity.

Also shown in each figure is the mean and standard deviation of the nondimensional residuals. As can be

seen, the mean of the u residuals is about four times larger than the mean of v and w. The nondimensional

mean uncertainty for v and w are seen to be very similar. Converting these means to dimensional quantities,

the u, v, and w uncertainty means are 20.94 m/s, 5.40 m/s, and 3.67 m/s, respectively. As expected, the

statistical uncertainty in u is larger than the uncertainty in v because (a) u is a factor of 10 larger than v,

and (b) u is the velocity component that is normal to the spanwise plane. It is well known in stereo-PIV

measurements that the uncertainty of the velocity component normal to the measurement plane is typically

two to three times larger than velocity components measured in the plane. The mean uncertainty for v is

about twice the estimate of the repeatability uncertainty quoted above (Sec. B). Note that even though the

w velocity component is normal to the streamwise plane, its mean uncertainty is slightly smaller than the

v component. The v component is the only component that occurs in both the streamwise and spanwise

planes.

A similar set of residual plots was generated for each of the measured quantities of interest. Here we

only show three samples of these results. Figure 6(a) shows the residuals of the streamwise normal stress

< u02 > /U2

∞ versus the normal stress measured at the intersection of each of the planes. Here we observe

an increase in the uncertainty as a function of the magnitude of the stress. This type of scatter plot for

uncertainty can be represented as a linear regression function. We require that the linear regression pass

through (0, 0) so that the only parameter that is computed is the slope of the line.

Figure 6(b) shows the residuals in the transverse normal stress < v02 > /U2

∞ versus the magnitude of

the average values of < v02 > /U2

∞ in each plane. Here we have a similar linear function to that seen for

< u02 > /U2

∞. The linear regression indicates that both the uncertainties in < u02 > /U2∞ and < v02 > /U2∞

increase with their magnitude. The slope gives an indication of relative magnitude of the uncertainty with

respect to the value of the quantity and can be converted into a percentage. We see then that this indicates

the uncertainty in < u02 > /U2

∞ relative to its magnitude is just under 20%, while for < v02 > /U2∞ it is

just under 40%. Figure 6(c) shows < u0v

0 > /U2

∞ versus the average of < u0v

0 > /U2

∞ measured in each

planes. It is not clear that the data support the idea of a linear variation for < u0v

0 > /U2

∞, but if a similar

relationship were assumed it would be of like magnitude. It is important to state that this level of uncertainty

in turbulence quantities provides a good qualitative measure for turbulence modelers.

Table 2 summarizes the results of the statistical uncertainty estimation for a number of quantities of

interest. For quantities of interest that show no clear trend of the scatter plot, the table shows values for

the mean and standard deviation of the residuals. For quantities that show a linearly increasing trend of

the uncertainty, the table shows the value of the slope of the linear regression (Y = a + bX). We are still

analyzing the results to try and explain why certain quantities have a linear trend and some do not. Values of

uncertainty (the mean of the residuals) that are given in the table are used for the experimental uncertainty

when comparing CFD results with the UMich. data.

NASA/TM—2011-216832 7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Average U Velocity, U/U !

U Velocity Residual, 

"U/U

!

# = 12.0o

# = 10.0o

# = 7.75o

Mean = 0.03473

$ = 0.02745 (a) Streamwise velocity, u/U∞

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

Average V Velocity, V/U !

V Velocity Residual, 

"V/U

!

# = 12.0o

# = 10.0o

# = 7.75o

Mean = 8.947E−3

$ = 7.002E−3

(b) Transverse velocity, v/U∞

0 0.002 0.004 0.006 0.008 0.01 0.012 0.014 0

0.005

0.01

0.015

0.02

0.025

0.03

Average W Velocity, W/U !

W Velocity Residual, 

"W/U

!

# = 12.0o

# = 10.0o

# = 7.75o

Mean = 6.081E−3

$ = 4.753E−3

(c) Spanwise velocity, w/U∞

Figure 5. Residuals of multiple measurements velocity and mean velocity components from the UMich. experiment

NASA/TM—2011-216832 8
0 0.002 0.004 0.006 0.008 0.01 0.012 0.014 0.016 0.018 0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

Average Streamwise Normal Stress, <u’2>/U!

2

Streamwise Normal Stress Residual, 

"<u’>

2/U

!

2

# = 12.0o

# = 10.0o

# = 7.75o

Linear Fit

Mean = 6.334E−4

Slope = 0.1978

$ = 1.087E−4

R2

 = 0.4077

(a) Streamwise normal stress, < u02 > /U2∞

0 0.2 0.4 0.6 0.8 1 1.2 1.4

x 10−3

0

1

2

3

4

5

6

7

8 x 10−4

Average Transverse Normal Stress, <v’ 2>/U!

2

Transverse Normal Stress Residual, 

"<v’

2>/U

!

2

# = 12.0o

# = 10.0o

# = 7.75o

Linear Fit

Mean = 2.009E−4

Slope = 0.3804

$ = 1.426E−4

R2

 = 0.3643

(b) Transverse normal stress, < v02 > /U2∞

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

x 10−3

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1 x 10−3

Average U−V Shear Stress, (U’V’)/U!

2

U

−V Shear Stress Residual, 

"(U’V’)/U

!

2

# = 12.0o

# = 10.0o

# = 7.75o

Mean = 5.317E−5

$ = 6.471E−5

(c) Shear stress, < u0v

0 > /U2∞

Figure 6. Residuals of turbulent stresses versus the average stress measured in each plane from the UMich.

experiment

NASA/TM—2011-216832 9
u/U∞ v/U∞ w/U∞ < u02 > /U2

∞ < v02 > /U2∞ < u0v

0 > /U2

∞

mean uncertainty 0.03473 0.008947 0.006081 6.334·10−4 2.009·10−4 5.317·10−5

std. dev. of the uncertainty 0.02745 0.007002 0.004753 0.001087 1.426·10−4 6.471·10−5

linear regression slope, b 0.1978 0.3804

Table 2. Summary of the statistical uncertainty analysis

IV. CFD Solutions

Eleven individuals from nine different organizations submitted solutions to the workshop for evaluation. Table 3 summarizes the submissions. The solutions were from both Reynolds Averaged Navier-Stokes

(RANS) methods and unsteady Navier-Stokes methods, which we will classify here as large-eddy simulation

(LES). The RANS methods employed a variety of turbulence models and the LES methods used a variety of

sub-grid models and numerical schemes. Both structured and unstructured grid methods were employed.

Name Organization Code Method Turbulence Model

Baurle NASA Langley Vulcan RANS Menter BSL

Baurle NASA Langley Vulcan RANS Menter SST

Baurle NASA Langley Vulcan RANS k − ω

Baurle NASA Langley Vulcan RANS k − ω + Durbin correction

Bhagwandin Army Research Lab CFD++ RANS Realizable k − 

Bhagwandin Army Research Lab CFD++ RANS Spalart-Almaras

Bhagwandin Army Research Lab CFD++ RANS Menter SST

Duraisamy Stanford Univ. Joe RANS Menter SST

Edwards North Carolina State Univ. LES

Edwards North Carolina State Univ. RANS Menter SST

Georgiadis NASA Glenn Wind-US RANS Menter SST

Jamalamadaka Michigan State Univ. LES

Park15 NASA Langley FUN3D RANS Spalart-Almaras

Pirozzoli16 Univ. Roma LES

Rumsey NASA Langley CFL3D RANS Spalart-Almaras

Subbareddy Univ. of Minnesota US3D LES IDDES

Tapee Northrup-Grumman RANS

Table 3. Summary of the submitted solutions

The majority of RANS submissions modeled the entire wind tunnel from the nozzle plenum to downstream

of the test section. In these cases the tunnel stagnation temperature and pressure were specified in the

plenum. The LES calculations typically used a RANS simulation as an initial solution and to provide

a mean inflow profile to the unsteady simulation of the test section. The LES solutions also required a

fluctuating component to the inflow profile. This was obtained through boundary layer recycling techniques,

precursor simulations or theoretical means.

Each analyst was responsible for post-processing their solutions. This included interpolating their solutions onto the experimental measurement grids and ensuring that the quantities were provided in the proper

dimensional form. The analysts were free to choose which cases to run and what data to provide. A total of

36 solutions were provided and 21 included the turbulent stresses.

NASA/TM—2011-216832 10
A. Description of the Error Metric

In order to objectively evaluate the submitted solutions an error metric, E, was defined. The validation

metric is a simple average of the magnitude of the difference between the CFD and experiment at every

point in a measurement plane. The metric is expressed as

E(f) = 1

N

X

N

n=1



(fcfd)n − (fexp)n



 (1)

For the error in velocities the metric is nondimensionalized by the freestream velocity, U∞. For the error

in the turbulence shear stress the metric is nodimensionalized by the square of the freestream velocity, U

2

∞.

The IUSTI normal stresses (√

< u02 > &

√

< v02 >) are nondimensionalized by U∞ and the UMich normal

stresss (< u02 > & < v02 >) are nomdimensionalized by U

2

∞ For a detailed discussion of the construction

and use of validation metrics see Ref.11

B. Examination of Representative Solutions

The error metric was computed for all the measured variables and all solutions. In the interest of brevity,

the comparisons shown here are limited to streamwise and transverse velocities as well as turbulent stresses.

The velocity derivatives from the UMich experiment were computed using the experimentally obtained

velocities and differenced on the PIV measurement grid. They do not provide any additional information for

comparison beyond the velocity data and are a poor comparison to the CFD because the CFD derivatives

were computed on their own grids, which are much finer. The spanwise velocity, w/U∞, is also a poor

quantity for comparison to CFD on the centerline. The tunnel geometry is symmetric about this plane. The

experimental data reveals that the spanwise velocity was not zero, indicating asymmetry in the flow. CFD

can not reliably predict such asymmetries and in almost all cases will produce symmetric solutions (in many

cases symmetry was specified in the computation) with zero spanwise velocity.

Because the streamwise velocity, u/U∞, is the dominant component most of the attention will be focused

on it. First, the general characteristics of the flowfields and the error metrics will be presented. For each

case, the solutions with the lowest (”best”) and highest (”worst”) measure of error in streamwise velocity,

E(u)/U∞ will be shown.

C. IUSTI Mach 2.25 – 8.0 degree case

Figure 7 compares contours of streamwise velocity for the experiment and the best and worst CFD predictions

of the IUSTI Mach 2.25, θ = 8.0

◦

case. For this case, the measurement plane extends vertically only to the

edge of the interaction. The shock wave is not visible. The flow separates from the wall and the boundary

layer downstream of the separation bubble is significantly thicker. The best solution appears very similar to

the experiment, although the size of the separation bubble and downstream boundary layer are slightly underpredicted. The worst solution shows a much smaller separation and thinner downstream boundary layer.

Fig. 8 shows the contours of the error metric for the streamwise velocity. The error contours clearly show

the regions where the predictions deviate from the PIV data. The regions of high error are concentrated

near the separation bubble. The maximum error in the best solution is approximately 0.2U∞, while the

maximum error in the worst solution is near 0.35U∞. A more traditional comparison is shown in Fig. 9

where velocity are given at several streamwise locations. Error bars based on the experimentalists’ estimate5

are included on the PIV data, but are within the size of the symbols representing the data. Despite the fact

that the PIV method can not measure very near the wall, it is seems apparent from the profiles that the

best solution should do a much better job of predicting quantities of interest such as skin friction.

D. UMich. Mach 2.75 – 7.75, 10.0 and 12.0 degree cases

Figure 10 compares contours of streamwise velocity for the experiment and the best and worst CFD predictions of the UMich Mach 2.75, θ = 7.75◦

case. For the Michigan cases, the measurement plane extends

beyond the interaction to include a portion of the freestream flow. The shock wave is visible, but appears

smeared in the data. This smearing could be due to some unsteadiness in the shock position and/or the

volume-averaging nature of the PIV technique. It is not clear if the flow is separated. Both CFD solutions

under-predict the size of the interaction region and downstream boundary layer thickness. Fig. 11 shows the

NASA/TM—2011-216832 11
(a) experiment

(b) ”best” solution

(c) ”worst” solution

Figure 7. IUSTI M = 2.25 θ = 8.0

◦, contours of streamwise velocity, u/U∞

(a) ”best” solution

(b) ”worst” solution

Figure 8. IUSTI M = 2.25 θ = 8.0

◦, contours of streamwise velocity error, E(u)/U∞

NASA/TM—2011-216832 12
u/U!

y (mm)

0 1 2 3 4 5 0

2

4

6

8

10

12

14

Expt.

"Best"

"Worst"

x = 240mm x = 300mm x = 320mm x = 360mm x = 400mm

Figure 9. IUSTI M = 2.25 θ = 8.0

◦, streamwise velocity profiles, u/U∞

contours of the error metric for the streamwise velocity. In addition to the error in the interaction region, the

plots show some error in the location of the shock wave and its reflection. The maximum error in the best

solution is approximately 0.13U∞, while the maximum error in the worst solution is near 0.27U∞. Velocity

profiles are shown in Fig. 12. Error bars based on the analysis in this paper are included on the PIV data.

Equivalent figures for the UMich θ = 10.0

◦ and θ = 12.0◦ are shown in Figs. 13 - 15 and 16 - 18. The

interaction grows larger with increasing deflection angle and the error levels increase for both the best and

worst predicted solutions.

E. Prediction of Turbulent Stresses

The turbulent stresses are computed very differently depending on the numerical approach used. For the

RANS solutions submitted, the stresses are not explicitly computed and must be derived from the turbulence

model variables. Typically the Boussinesq approximation is used to compute the components of the stress

tensor based on the turbulent kinetic energy from a two-equation model. In the case of a one-equation

turbulence model, an additional approximation for turbulent kinetic energy must be made. An example

of how this may be done is given by Georgiadis et al.17 For the LES methods, the stresses are computed

directly in the simulation. Because the LES solutions have less reliance on modeling approximations, one

can expect that they will yield better results.

Two predictions for the turbulent stresses were submitted for the IUSTI case, one RANS and one LES.

The solutions are compared with the experiment in Figs. 19 - 21. The experimental data shows very high

levels of streamwise normal stress in the forward portion of the interaction region. This feature is predicted

well by the LES and missed by the RANS (Fig. 19).

The experiment shows that the form and levels of the transverse normal stress differ significantly from

the streamwise normal stress. The LES solution depicts this change accurately. The RANS prediction of

the two normal stresses are very similar. This deficiency in the RANS prediction is a direct consequence of

the approximate method necessary to derive the stresses from the turbulence model.

The shear stress is shown in Fig. 21. Here, the RANS solution is much improved. The improved prediction

of the shear stress for RANS is not surprising. RANS turbulence models directly affect the solution through

the shear stress term in the equations and the models are developed and tuned to accurately model this

term.

NASA/TM—2011-216832 13
(a) experiment

(b) ”best” solution (c) ”worst” solution

Figure 10. UMich. M = 2.75 θ = 7.75◦, contours of streamwise velocity, u/U∞

(a) ”best” solution (b) ”worst” solution

Figure 11. UMich. M = 2.75 θ = 7.75◦, contours of streamwise velocity error, E(u)/U∞

NASA/TM—2011-216832 14
u/U!

y(mm)

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0

5

10

15

20 Expt.

"Best"

"Worst"

x = 20.76mm x = 34.63mm x = 53.63mm x = 30.53mm x = 41.82mm

Figure 12. UMich. M = 2.75 θ = 7.75◦, streamwise velocity profiles, u/U∞

(a) experiment

(b) ”best” solution (c) ”worst” solution

Figure 13. UMich. M = 2.75 θ = 10.0

◦, contours of streamwise velocity, u/U∞

NASA/TM—2011-216832 15
(a) ”best” solution (b) ”worst” solution

Figure 14. UMich. M = 2.75 θ = 10.0

◦, contours of streamwise velocity error, E(u)/U∞

u/U!

y (mm)

0 1 2 3 4 5 0

5

10

15

20

Expt.

"Best"

"Worst"

x = 27.77mm x = 38.26mm x = 41.59mm x = 46.36mm x = 55.89mm Figure 15. UMich. M = 2.75 θ = 10.0

◦, streamwise velocity profiles, u/U∞

NASA/TM—2011-216832 16
(a) experiment

(b) ”best” solution (c) ”worst” solution

Figure 16. UMich. M = 2.75 θ = 12.0

◦, contours of streamwise velocity, u/U∞

(a) ”best” solution (b) ”worst” solution

Figure 17. UMich. M = 2.75 θ = 12.0

◦, contours of streamwise velocity error, E(u)/U∞

NASA/TM—2011-216832 17
u/U!

y (mm)

0 1 2 3 4 5 0

5

10

15

20

Expt.

"Best"

"Worst"

x = 20.47mm x = 55.62mm x = 33.91mm x = 39.08mm x = 45.28mm

Figure 18. UMich. M = 2.75 θ = 12.0

◦, streamwise velocity profiles, u/U∞

(a) experiment

(b) RANS solution

(c) LES solution

Figure 19. IUSTI M = 2.25 θ = 8.0

◦, contours of streamwise normal stress, √

< u02 >/(U∞)

2

NASA/TM—2011-216832 18
(a) experiment

(b) RANS solution

(c) LES solution

Figure 20. IUSTI M = 2.25 θ = 8.0

◦, contours of transverse normal stress, √

< v02 >/(U∞)

2

(a) experiment

(b) RANS solution

(c) LES solution

Figure 21. IUSTI M = 2.25 θ = 8.0

◦, contours of shear stress, − < u0v0 > /(U∞)2

NASA/TM—2011-216832 19
F. Comparison of all Solutions

1. IUSTI, Mach 2.25, θ = 8.0

The error for all seven predictions (denoted A through G) of the IUSTI case are shown in Fig. 22. For the

velocity components, the experimental uncertainty quoted by the experimentalists, 1.1 percent of freestream

velocity, is represented by the dashed line. The average error in streamwise velocity for the CFD predictions

ranged between 3.8 and 9.0 percent of freestream velocity, which is well above the experimental error. The

errors in transverse velocity are all near the experimental uncertainty. The relative accuracy of the methods

is not consistent between the variables plotted here. For example, submission F provides the lowest error in

streamwise velocity, but provides the highest error in transverse velocity. The error in the prediction of the

turbulent stresses corresponds to the discussion in section E above. The LES prediction, D, does a superior

job for the normal stresses and the RANS prediction, G, is better for the shear stress.

0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

0.07 

0.08 

0.09 

0.1 

Error in u 

A 

B 

C 

D 

E 

F 

G 

(a) error in streamwise velocity, E(u)/U∞

0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

0.07 

0.08 

0.09 

0.1 

Error in v 

A 

B 

C 

D 

E 

F 

G 

(b) error in transverse velocity, E(v)/U∞

0.00E+00 

1.00E-05 

2.00E-05 

3.00E-05 

4.00E-05 

5.00E-05 

Error in sqrt<u'**2> 

D 

G 

(c) error in streamwise normal stress, E(

√

< u02 >)/U∞

0.00E+00 

1.00E-05 

2.00E-05 

3.00E-05 

4.00E-05 

5.00E-05 

Error in sqrt<v'**2> 

D 

G 

(d) error in transverse normal stress, E(

√

< v02 >)/U∞

0.00E+00 

1.00E-04 

2.00E-04 

3.00E-04 

4.00E-04 

5.00E-04 

6.00E-04 

Error in -<u'v'> 

D 

G 

(e) error in shear stress, E(− < u0v

0 >)/U2∞

Figure 22. Error assessment of the IUSTI Mach 2.25, θ = 8.0

◦ case

NASA/TM—2011-216832 20
2. UMich. Mach 2.75 – 7.75, 10.0 and 12.0 degree cases

The errors for all predictions of the UMich., Mach 2.75 cases are shown in Figs. 23 - 25. The experimental

uncertainties from the statistical estimation are represented by the dashed lines. For the turbulent normal

stresses, the mean uncertainty was used, which represents the uncertainty in all planes. A better estimate

for a given plane could be made using the linear regression and integrating the product of the slope and

measured value of the stress over that plane. All of the CFD predictions of streamwise velocity and streamwise

normal stress are near (many are below) the levels of the experimental uncertainty. This suggests that any

conclusions based on the error data are not valid. But as shown in Figs. 12, 15 and 18, in the interaction

region the predictions lie outside of the experimental error bars suggesting that the use of this error metric

with the current data can still be useful.

The computed errors in streamwise velocity for the CFD predictions are given in Table 4. The errors

grow with increasing deflection angle. This is expected and is a result of the stronger shock wave, larger

interaction region and more complex flowfield. The lower average errors compared to the IUSTI case is due

to the inclusion of a portion of the freestream flow in the plane of interest. The freestream portion of the

flow is much better predicted than the interaction region. Including this portion of the flowfield in the error

calculation results in a lower average error (Figs. 11, 14 & 17).

As with the IUSTI case, there does not appear to be a correlation between prediction methodologies and

the relative error for each variable examined.

Deflection Angle Min. E(u)/U∞ Max. E(u)/U∞

7.75 0.0240 0.0404

10.0 0.0341 0.0463

12.0 0.0422 0.0581

Table 4. Minimum and maximum error in streamwise velocity for the UMich. cases

G. Discussion of Trends in the Data

In general the errors in the CFD solutions all fell within a relatively narrow band and no one method

predicted all variables well. No method stood out as being clearly superior to the others. However the error

contours and velocity profiles indicate that there is room for improvement in the modeling. RANS methods,

such as Reynolds stress and algebraic Reynolds stress models may offer improved predictions of the turbulent

stresses and other flow quantities. Also, the LES predictions suggest that as the method matures results will

improve.

The submitted solutions provided a wealth of information. But, because each analyst used a different

approach there are a large number of variables to consider when attempting to analyze the results. These

variables include; numerical scheme, grid resolution, grid type, boundary conditions, turbulence model, gas

model, etc. Based on the information that was submitted with the results and presented at the workshop an

attempt was made to discern trends in the error data. This is by no means a complete analysis. And it is

suggested that in order to do a more thorough study, future efforts should include submitted solutions that

employ common elements such as grids, numerical schemes, etc.

1. Effect of RANS Turbulence Model

The clearest discernable effect on the solutions for the RANS submissions was that of the turbulence model.

There were seven RANS methodologies that were run for all three UMich. flow cases. Fig. 26 shows the

error in the streamwise velocity prediction for these submissions. For the 7.75 degree case all the Menter

SST, k−ω and Spalart-Almaras model solutions are within 0.5 percent of each other. At the larger deflection

angles, the error in all of the solutions increases and a clear trend is seen. All of the methods show the same

relative error levels for the 10 and 12 degree cases. The k−ω models provide the lowest error and the Menter

BSL model has the highest error. The Menter SST and Spalart-Almaras models produce similar error levels

in between the minimum and maximum levels. For the 7.75 degree case several models also exhibit the same

behavior in the relative error. The error for two Menter SST, the Spalart-Almaras and the Menter BSL

NASA/TM—2011-216832 21
0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

Error in u 

B 

F 

G 

H 

I 

J 

K 

L 

M 

N 

O 

P 

Q 

(a) error in streamwise velocity, E(u)/U∞

0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

Error in v 

B 

F 

G 

H 

I 

J 

K 

L 

M 

N 

O 

P 

Q 

(b) error in transverse velocity, E(v)/U∞

0.00E+00 

2.00E-04 

4.00E-04 

6.00E-04 

8.00E-04 

1.00E-03 

1.20E-03 

1.40E-03 

1.60E-03 

Error in <u'**2> 

G 

H 

J 

M 

N 

O 

P 

(c) error in streamwise normal stress, E(< u02 >)/U2∞

0.00E+00 

2.00E-04 

4.00E-04 

6.00E-04 

8.00E-04 

1.00E-03 

1.20E-03 

1.40E-03 

1.60E-03 

Error in <v'**2> 

G 

H 

J 

M 

N 

O 

P 

(d) error in transverse normal stress, E(< v02 >)/U2∞

0.00E+00 

2.00E-04 

4.00E-04 

6.00E-04 

8.00E-04 

1.00E-03 

1.20E-03 

1.40E-03 

1.60E-03 

Error in <u'v'> 

G 

H 

J 

M 

N 

O 

P 

(e) error in shear stress, E(< u0v

0 >)/U2∞

Figure 23. Error assessment of the UMich. Mach 2.75, θ = 7.75◦ case

NASA/TM—2011-216832 22
0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

Error in u 

B 

G 

H 

I 

M 

N 

O 

P 

Q 

(a) error in streamwise velocity, E(u)/U∞

0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

Error in v 

B 

G 

H 

I 

M 

N 

O 

P 

Q 

(b) error in transverse velocity, E(v)/U∞

0.00E+00 

2.00E-04 

4.00E-04 

6.00E-04 

8.00E-04 

1.00E-03 

1.20E-03 

1.40E-03 

1.60E-03 

Error in <u'**2> 

G 

H 

M 

N 

O 

P 

(c) error in streamwise normal stress, E(< u02 >)/U2∞

0.00E+00 

2.00E-04 

4.00E-04 

6.00E-04 

8.00E-04 

1.00E-03 

1.20E-03 

1.40E-03 

1.60E-03 

Error in <v'**2> 

G 

H 

M 

N 

O 

P 

(d) error in transverse normal stress, E(< v02 >)/U2∞

0.00E+00 

2.00E-04 

4.00E-04 

6.00E-04 

8.00E-04 

1.00E-03 

1.20E-03 

1.40E-03 

1.60E-03 

Error in <u'v'> 

G 

H 

M 

N 

O 

P 

(e) error in shear stress, E(< u0v

0 >)/U2∞

Figure 24. Error assessment of the UMich. Mach 2.75, θ = 10.0

◦ case

NASA/TM—2011-216832 23
0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

Error in u 

B 

G 

H 

M 

N 

O 

P 

(a) error in streamwise velocity, E(u)/U∞

0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

Error in v 

B 

G 

H 

M 

N 

O 

P 

(b) error in transverse velocity, E(v)/U∞

0.00E+00 

2.00E-04 

4.00E-04 

6.00E-04 

8.00E-04 

1.00E-03 

1.20E-03 

1.40E-03 

1.60E-03 

Error in <u'**2> 

G 

H 

M 

N 

O 

P 

(c) error in streamwise normal stress, E(< u02 >)/U2∞

0.00E+00 

2.00E-04 

4.00E-04 

6.00E-04 

8.00E-04 

1.00E-03 

1.20E-03 

1.40E-03 

1.60E-03 

Error in <v'**2> 

G 

H 

M 

N 

O 

P 

(d) error in transverse normal stress, E(< v02 >)/U2∞

0.00E+00 

2.00E-04 

4.00E-04 

6.00E-04 

8.00E-04 

1.00E-03 

1.20E-03 

1.40E-03 

1.60E-03 

Error in <u'v'> 

G 

H 

M 

N 

O 

P 

(e) error in shear stress, E(< u0v

0 >)/U2∞

Figure 25. Error assessment of the UMich. Mach 2.75, θ = 12.0

◦ case

NASA/TM—2011-216832 24
predictions behave similarly to the 10 and 12 degree cases. The k −ω and one of the Menter SST predictions

do not follow the trend.

A closer examination of the submitted solutions reveals that the predictions are strongly determined by

the turbulence model used. This is illustrated by: 1) The three Menter SST solutions were from different

submissions and used different grids and codes, yet there are only slight differences between them; 2) One

of the Menter SST solutions, the k − ω solutions and the Menter BSL solution were all obtained using the

same grid and code, illustrating a strong dependence on turbulence model.

0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

Error in u 

SST 

SST 

SST 

k-omega 

k-omega + 

SA 

BSL 

(a) UMich. Mach 2.75, θ = 7.75

0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

Error in u 

SST 

SST 

SST 

k-omega 

k-omega+ 

SA 

BSL 

(b) UMich. Mach 2.75, θ = 10.0

0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

Error in u 

SST 

SST 

SST 

k-omega 

k-omega+ 

SA 

BSL 

(c) UMich. Mach 2.75, θ = 12.0

Figure 26. RANS turbulence model comparison for the UMich. Mach 2.75 cases. Error in streamwise velocity,

E(u)/U∞

2. Effect of RANS versus LES

The UMich. Mach 2.75, θ = 7.75 case was the only case where multiple LES solutions were submitted. The

LES solutions were the best and worst predictions for this case. Fig. 27 shows the error levels for streamwise

and transverse velocity for the best and worst RANS and LES predictions. Both methods produce very

similar error levels. But because RANS methods are far more mature than LES, there is still potential for

LES to provide superior predictions. In addition, if the prediction of normal stresses is important, LES

should provide better accuracy.

3. Effect of Structured versus Unstructured Grids

Unstructured grid codes are widely used today. Fig. 28 shows the predictions for the IUSTI case grouped by

structured and unstructured methods. A similar spread in error levels is seen for both methods indicating

that there is no discernible difference between the grid types.

NASA/TM—2011-216832 25
0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

Error in u 

Best RANS 

Worst RANS 

Best LES 

Worst LES 

(a) error in streamwise velocity, E(u)/U∞

0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

Error in v 

Best RANS 

Worst RANS 

Best LES 

Worst LES 

(b) error in transverse velocity, E(v)/U∞

Figure 27. RANS/LES comparison for the UMich. Mach 2.75, θ = 7.75◦ case

0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

0.07 

0.08 

0.09 

0.1 

Error in u 

Structured 

Structured 

Structured 

Unstructured 

Unstructured 

Unstructured 

Unstructured 

(a) error in streamwise velocity, E(u)/U∞

0 

0.01 

0.02 

0.03 

0.04 

0.05 

0.06 

0.07 

0.08 

0.09 

0.1 

Error in v 

Structured 

Structured 

Structured 

Unstructured 

Unstructured 

Unstructured 

Unstructured 

(b) error in transverse velocity, E(v)/U∞

Figure 28. Structured/unstructured grid comparison for the IUSTI Mach 2.25, θ = 8.0

◦ case

NASA/TM—2011-216832 26
V. Summary and Conclusions

A workshop on the computational fluid dynamics (CFD) prediction of shock boundary-layer interactions

(SBLIs) was held at the 48th AIAA Aerospace Sciences Meeting in Orlando, Florida. The purpose of

the workshop was to share prediction methodologies, assess the state of the art in SBLI prediction and

determine the most promising methods. Prior to the meeting four test cases were identified and contributions

of solutions were solicited from the aerospace community. A total of 36 solutions from 11 contributors

and 9 organizations were submitted. As part of this effort, an assessment of the submitted solutions was

performed. This consisted of an assessment of the experimental data, the development of an error metric

and the assessment of the CFD data using the error metric.

The experimental data for the four test cases came from two different sources: the Institut Universitaire

des Syst`emes Thermiques Industriels (IUSTI) in Marseilles, France and the University of Michigan (UMich.).

The data used for comparison consisted of velocities and turbulent stresses measured on a plane parallel

with the freestream flow and the tunnel sidewall. An uncertainty analysis of the experimental data had been

previously conducted by each of the research groups involved. As part of this work a statistical uncertainty

analysis of the UMich. data was performed. The analysis compared data from the streamwise and spanwise

measurement planes at the locations where the planes intersected. Bi-linear interpolation was used to put

the data at common points in space. The residuals for all the measured quantities were computed and

the mean values of those residuals is used for the uncertainty estimate. This analysis showed the largest

uncertainty was for the streamwise velocity and was (in the mean) 3.5 percent of the freestream velocity.

The uncertainties in the turbulent normal stresses were found to increase linearly with the magnitude of the

stress.

An error metric that represents the average error in the predicted quantity for the entire data plane was

defined. This error metric was used to assess the relative quality of each of the submitted CFD solutions.

Several variables were examined; streamwise velocity, u, transverse velocity, v, and turbulent stresses, <

u

02 >, < v02 > and < u0v0 >. It was found that the relative accuracy of the methods was not consistent

between the variables of interest (ie. a good prediction of u did not imply a good prediction of v or any

other variable for a given method). Thus, the streamwise velocity was chosen to assess the computational

methods. For the most part, the submitted solutions were remarkably similar in the error levels. The primary

differences between solutions were the details of the boundary-layer separation. Because of the many different

methods used, the lack of consistency for the predictions and the similarity of the solutions, it was difficult to

identify clear trends or single out any method as superior. In general the errors for all the solutions increased

with increasing shock strength (increased deflection angle). The choice of turbulence model appears to be

the dominant factor in the accuracy of the Reynolds Averaged Navier-Stokes (RANS) solutions. Large-eddy

simulations had similar error levels as the RANS methods, but provided superior predictions of the normal

stresses. There appeared to be no dependence on computational grid type (structured or unstructured).

The data shows that there is a need for additional research in the prediction of SBLIs. First, high quality

experimental data that includes the effects of the sidewall and corner flow interactions is needed and the

uncertainty of the data should be quantified. RANS turbulence models that provide normals stresses such as

full Reynolds stress and algebraic Reynolds stress models should be investigated for this problem. Finally,

LES methods should continue to be advanced as they show promise for this type of flow.

References

1Delery, J. and Marvin, J. G., “Shock-Wave Boundary Layer Interactions,” AGARDograph No. 280, 1986, E. Reshotko

(ed.).

2Benek, J., “Overview of the 2010 AIAA Shock Boundary Layer Interaction Workshop,” AIAA Paper 2010-4821, 2010.

3Hirsch, C., “SBLI Lessons Learned - CFD Simulations of Two Test Cases ,” AIAA Paper 2010-4824, 2010.

4Benek, J. and Babinsky, H., “Lessons Learned from the 2010 AIAA Shock Boundary Layer Interaction Workshop,” AIAA

Paper 2010-4825, 2010.

5Dupont, P., Piponniau, S., Sidorenko, A., and Debieve, J., “Investigation by Particle Image Velocimetry Measurements

of Oblique Shock Reflection with Separation,” AIAA Journal, Vol. 46, No. 6, 2008, pp. 1365–1370.

6Unsteady Effects in Shock Wave Induced Separation, edited by P. Doerffer, C. Hirsch, J.-P. Dussauge, H. Babinsky, and

G. Barakos, Springer, 2010.

7Lapsa, A. P. and Dahm, W. J. A., “Stereo Particle Image Velocimetry of Nonequilibrium Turbulence Relaxation in a

Supersonic Boundary Layer,” Experiments in Fluids, 2010, online June 4, 2010.

8Dussauge, J. P., Debieve, J. F., Dupont, P., Piponniau, S., and Souverein, L. J., “Final Report on the Measurements of

Shock Reflection at M=2.25,” UFAST, IUSTI Deliverable No. 3.3.5, 2009.

NASA/TM—2011-216832 27
9Oberkampf, W. L. and Aeschliman, D. P., “Joint Computational/Experimental Aerodynamics Research on a Hypersonic

Vehicle: Part 1, Experimental Results,” AIAA Journal, Vol. 30, No. 8, 1992.

10Oberkampf, W. L., Aeschliman, D. P., Henfling, J. F., and Larson, D. E., “Surface Pressure Measurements for CFD Code

Validation in Hypersonic Flow.” AIAA Paper 95-2273, 1995.

11Oberkampf, W. L. and Roy, C. J., Verification and Validation in Scientific Computing, Cambridge University Press,

2010.

12Montgomery, D. C., Design and Analysis of Experiments, John Wiley, 5th ed., 2000.

13Montgomery, D. C., Statistics for Experimenters: Design, Innovation, and Discovery, John Wiley, 2nd ed., 2005.

14Coleman, H. W. and Jr., W. G. S., Experimentation and Uncertainty Analysis for Engineers, John Wiley, 2nd ed., 1999.

15Park, M. A. and Carlson, J.-R., “Turbulent Output-Based Anisotropic Adaptation,” AIAA Paper 2010-168, 2010.

16Pirozzoli, S. and Grasso, F., “Direct numerical simulation of impinging shock wave/turbulent boundary layer interaction

at M = 2.25,” Physics of Fluids, Vol. 18, No. 065113, 2006.

17Georgiadis, N. J., Rumsey, C. L., Yoder, D. A., and Zaman, K. B. M. Q., “Turbulence Modeling Effects on Calculation

of Lobed Nozzle Flowfields,” Journal of Propulsion and Power, Vol. 22, No. 3, 2006, pp. 567–575.

NASA/TM—2011-216832 28
REPORT DOCUMENTATION PAGE Form Approved 

OMB No. 0704-0188 

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the 

data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this 

burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. 

Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB 

control number. 

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 

1. REPORT DATE (DD-MM-YYYY)

01-03-2011 

2. REPORT TYPE

Technical Memorandum 

3. DATES COVERED (From - To)

4. TITLE AND SUBTITLE

Assessment of Computational Fluid Dynamics (CFD) Models for Shock Boundary-Layer 

Interaction 

5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S)

DeBonis, James, R.; Oberkampf, William, L.; Wolf, Richard, T.; Orkwis, Paul, D.; Turner, 

Mark, G.; Babinsky, Holger 

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

WBS 984754.02.07.03.13.02 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration 

John H. Glenn Research Center at Lewis Field 

Cleveland, Ohio 44135-3191 

8. PERFORMING ORGANIZATION

 REPORT NUMBER

E-17462 

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration 

Washington, DC 20546-0001 

10. SPONSORING/MONITOR'S

 ACRONYM(S)

NASA 

11. SPONSORING/MONITORING

 REPORT NUMBER

NASA/TM-2011-216832 

12. DISTRIBUTION/AVAILABILITY STATEMENT

Unclassified-Unlimited 

Subject Categories: 02 and 07 

Available electronically at http://www.sti.nasa.gov 

This publication is available from the NASA Center for AeroSpace Information, 443-757-5802

13. SUPPLEMENTARY NOTES

14. ABSTRACT

A workshop on the computational fluid dynamics (CFD) prediction of shock boundary-layer interactions (SBLIs) was held at the 48th AIAA 

Aerospace Sciences Meeting. As part of the workshop numerous CFD analysts submitted solutions to four experimentally measured SBLIs. 

This paper describes the assessment of the CFD predictions. The assessment includes an uncertainty analysis of the experimental data, the 

definition of an error metric and the application of that metric to the CFD solutions. The CFD solutions provided very similar levels of error 

and in general it was difficult to discern clear trends in the data. For the Reynolds Averaged Navier-Stokes methods the choice of turbulence 

model appeared to be the largest factor in solution accuracy. Large-eddy simulation methods produced error levels similar to RANS methods 

but provided superior predictions of normal stresses. 

15. SUBJECT TERMS

Shock waves; Turbulent boundary layer; Computational fluid dynamics 

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF

 ABSTRACT

UU 

18. NUMBER

 OF 

 PAGES

34 

19a. NAME OF RESPONSIBLE PERSON

STI Help Desk (email:help@sti.nasa.gov) 

a. REPORT

U 

b. ABSTRACT

U 

c. THIS 

PAGE

U 

19b. TELEPHONE NUMBER (include area code)

443-757-5802 

Standard Form 298 (Rev. 8-98)

Prescribed by ANSI Std. Z39-18