Shantaram S. Pai

Glenn Research Center, Cleveland, Ohio

David S. Riha

Southwest Research Institute, San Antonio, Texas

Model Verifi cation and Validation Concepts for a 

Probabilistic Fracture Assessment Model to Predict 

Cracking of Knife Edge Seals in the Space Shuttle 

Main Engine High Pressure Oxidizer

NASA/TM—2013-217849

April 2013
NASA STI Program . . . in Profi le

Since its founding, NASA has been dedicated to the 

advancement of aeronautics and space science. The 

NASA Scientifi c and Technical Information (STI) 

program plays a key part in helping NASA maintain 

this important role.

The NASA STI Program operates under the auspices 

of the Agency Chief Information Offi cer. It collects, 

organizes, provides for archiving, and disseminates 

NASA’s STI. The NASA STI program provides access 

to the NASA Aeronautics and Space Database and 

its public interface, the NASA Technical Reports 

Server, thus providing one of the largest collections 

of aeronautical and space science STI in the world. 

Results are published in both non-NASA channels 

and by NASA in the NASA STI Report Series, which 

includes the following report types:

• TECHNICAL PUBLICATION. Reports of 

completed research or a major signifi cant phase 

of research that present the results of NASA 

programs and include extensive data or theoretical 

analysis. Includes compilations of signifi cant 

scientifi c and technical data and information 

deemed to be of continuing reference value. 

NASA counterpart of peer-reviewed formal 

professional papers but has less stringent 

limitations on manuscript length and extent of 

graphic presentations.

• TECHNICAL MEMORANDUM. Scientifi c 

and technical fi ndings that are preliminary or 

of specialized interest, e.g., quick release 

reports, working papers, and bibliographies that 

contain minimal annotation. Does not contain 

extensive analysis.

• CONTRACTOR REPORT. Scientifi c and 

technical fi ndings by NASA-sponsored 

contractors and grantees.

• CONFERENCE PUBLICATION. Collected 

papers from scientifi c and technical 

conferences, symposia, seminars, or other 

meetings sponsored or cosponsored by NASA.

• SPECIAL PUBLICATION. Scientifi c, 

technical, or historical information from 

NASA programs, projects, and missions, often 

concerned with subjects having substantial 

public interest.

• TECHNICAL TRANSLATION. Englishlanguage translations of foreign scientifi c and 

technical material pertinent to NASA’s mission.

Specialized services also include creating custom 

thesauri, building customized databases, organizing 

and publishing research results.

For more information about the NASA STI 

program, see the following:

• Access the NASA STI program home page at 

http://www.sti.nasa.gov

• E-mail your question to help@sti.nasa.gov

• Fax your question to the NASA STI 

Information Desk at 443–757–5803

• Phone the NASA STI Information Desk at

 443–757–5802

• Write to:

 STI Information Desk

 NASA Center for AeroSpace Information

 7115 Standard Drive

 Hanover, MD 21076–1320
Shantaram S. Pai

Glenn Research Center, Cleveland, Ohio

David S. Riha

Southwest Research Institute, San Antonio, Texas

Model Verifi cation and Validation Concepts for a 

Probabilistic Fracture Assessment Model to Predict 

Cracking of Knife Edge Seals in the Space Shuttle 

Main Engine High Pressure Oxidizer

NASA/TM—2013-217849

April 2013

National Aeronautics and

Space Administration

Glenn Research Center

Cleveland, Ohio 44135

Prepared for the

Turbo Expo 2012

sponsored by the American Society of Mechanical Engineers (ASME)

Copenhagen, Denmark, June 11–15, 2012
Acknowledgments

Many people contributed to the development of the probabilistic fracture assessment model to analyze cracking in the space 

shuttle main engine (SSME) high-pressure oxidizer turbopump (HPOTP) knife edge seals. The success of this effort relied on 

the experience, expertise, and weekly peer reviews of the models and data. We acknowledge the following for contributing to 

the success of this effort: Pratt & Whitney Rocketdyne: Ken Head, Kon Haake, Stuart Montgomery, and George Prueger; NASA 

Marshall Space Flight Center: Gregory Swanson, Phillip Allen, and Erika Mendoza; and Southwest Research Institute: R. Craig 

McClung, Michael P. Enright, Carl F. Popelar, Chris J. Waldhart, and Brian M. Gardner.

Available from

NASA Center for Aerospace Information

7115 Standard Drive

Hanover, MD 21076–1320

National Technical Information Service

5301 Shawnee Road

Alexandria, VA 22312

Available electronically at http://www.sti.nasa.gov

Trade names and trademarks are used in this report for identifi cation 

only. Their usage does not constitute an offi cial endorsement, 

either expressed or implied, by the National Aeronautics and 

Space Administration.

Level of Review: This material has been technically reviewed by technical management. 
NASA/TM—2013-217849 1 

Model Verification and Validation Concepts for a Probabilistic Fracture 

Assessment Model to Predict Cracking of Knife Edge Seals in the

Space Shuttle Main Engine High Pressure Oxidizer

Shantaram S. Pai

National Aeronautics and Space Administration

Glenn Research Center

Cleveland, Ohio 44135

David S. Riha

Southwest Research Institute

San Antonio, Texas 78228

Abstract

Physics-based models are routinely used to predict the 

performance of engineered systems to make decisions such as 

when to retire system components, how to extend the life of 

an aging system, or if a new design will be safe or available.

Model verification and validation (V&V) is a process to 

establish credibility in model predictions. Ideally, carefully 

controlled validation experiments will be designed and 

performed to validate models or submodels. In reality, time 

and cost constraints limit experiments and even model 

development. This paper describes elements of model V&V 

during the development and application of a probabilistic 

fracture assessment model to predict cracking in space shuttle 

main engine high-pressure oxidizer turbopump knife-edge 

seals. 

The objective of this effort was to assess the probability of 

initiating and growing a crack to a specified failure length in 

specific flight units for different usage and inspection 

scenarios. The probabilistic fracture assessment model 

developed in this investigation combined a series of 

submodels describing the usage, temperature history, flutter 

tendencies, tooth stresses and numbers of cycles, fatigue 

cracking, nondestructive inspection, and finally the 

probability of failure. The analysis accounted for unit-to-unit 

variations in temperature, flutter limit state, flutter stress 

magnitude, and fatigue life properties. The investigation 

focused on the calculation of relative risk rather than absolute 

risk between the usage scenarios. Verification predictions 

were first performed for three units with known usage and 

cracking histories to establish credibility in the model 

predictions. Then, numerous predictions were performed for 

an assortment of operating units that had flown recently or 

that were projected for future flights. Calculations were 

performed using two NASA-developed software tools: 

NESSUS® for the probabilistic analysis, and NASGRO® for 

the fracture mechanics analysis. The goal of these predictions 

was to provide additional information to guide decisions on 

the potential of reusing existing and installed units prior to the 

new design certification.

Introduction

Numerical models are routinely used to predict the 

performance of engineered systems. Government and industry 

now routinely rely on model predictions to make such 

decisions as when to retire system components, how to extend 

the life of an aging system, or if a new design will be safe or 

available. The validity of many models used to predict the 

performance of existing engineered systems have been 

assessed through historical data but this type of validation is 

not possible for new designs or designs used in different 

environments. The validation of new models using experiments 

becomes more difficult and costly as the complexity and 

reliability requirements increase. For example, a highly reliable 

aircraft engine component is difficult to test to failure under 

operating conditions due to the high reliability. In addition, it 

may be cost prohibitive to actually test an expensive 

component to failure. Other systems are impractical to test 

such as the in vivo measurement of performance measures in 

human subjects or systems in extreme environments. Valid 

model predictions become increasingly important as the cost, 

reliability, and experimental complexity for the engineered 

system increases. 

Model verification and validation (V&V) provides a 

systematic approach to establishing confidence in model 

predictions. Most models requiring this level of validation are 

complex one-of-a-kind problems and have different time and 

budget constraints. Experience has shown that a standard and 

consistent model V&V methodology may not be practical for 

these types of problems. This observation is supported by the 

fact that there are no general standards for model V&V at this 

time. However, there are several V&V guidelines produced by 

technical committees for computational fluids dynamics

(Ref. 1) and solids mechanics (Ref. 2). 

The goal of model V&V is to establish credibility in the 

model predictions that guide engineering decisions. In many 

cases the credibility is established in the engineering process. 

For example, models are developed by experienced analyst and 

compared to historical data or a single experiment. 

Conservative values of loads and material strengths may be 

used to mitigate uncertainties about these parts of the models. 
NASA/TM—2013-217849 2 

This process may also include design and/or peer reviews to 

further establish credibility in the model predictions. From a 

high level view, a formal model V&V effort would include 

validation experiments and metrics that quantify the 

agreement between model predictions and experiments. In 

practical applications, the validation experiments may be cost 

or time prohibitive and analysis must rely on historical 

performance information or qualification experiments for the 

comparison. In addition, there may be few experiments or 

limited historical data for a rigorous statistical quantitative

comparison between the model predictions and experiments. 

Finally, model approximations may be required along with 

engineering estimates of model parameters due to the time 

and cost constraints. These approximations, assumptions, and 

limited data lead to uncertainty in both the model predictions 

and experiments. Thus uncertainty quantification is an 

integral part of the model V&V process.

The goal of this paper is to describe the development of a

model that could credibly predict the performance of a 

complex engineered system for different operating conditions 

within a tight time and cost budget. The model was developed 

by experienced analysts, compared to historical data, and peer

reviewed frequently. While a formal V&V process was 

desired, and engineering approach was required to establish 

credibility in the model predictions. This paper captures some 

of the experience gained that can benefit future model V&V 

efforts.

Verification and Validation

Model verification and validation (V&V) is an enabling 

methodology for the development of computational models 

that can be used to make predictions with quantified

confidence. Model V&V procedures are needed to reduce the 

time, cost, and risk associated with component and full-scale 

testing of products, materials, and engineered systems. 

Quantifying the confidence and predictive accuracy of model 

calculations provides the decision-maker with the information 

necessary for making a risk-informed decision.

Model V&V is the primary process for quantifying and 

building credibility in computational models. Verification is 

the process of determining that a model implementation 

accurately represents the developer’s conceptual description 

of the model and its solution. Validation is the process of 

determining the degree to which a model is an accurate 

representation of the real world from the perspective of the 

intended uses of the model (Ref. 2). In short, verification is a 

mathematics issue, whereas validation is a physics issue.

Uncertainty quantification plays a key role in model V&V. 

There is uncertainty in the model predictions due to inherent 

and subjective uncertainties in the model. Likewise, the 

measurements that are made to validate these simulation 

outputs also contain errors and uncertainties. While the 

experimental outcome is used as the reference for 

comparison, the V&V process does not presume the 

experiment to be more accurate than the simulation. Instead, 

the goal is to quantify the uncertainties in both experimental 

and simulation results such that the model fidelity requirements 

can be assessed (validation) and the predictive accuracy of the 

model quantified.

Uncertainty Quantification 

Uncertainty quantification in the context of model V&V is 

use to explicitly account for errors and variations in the model 

and model input parameters and quantify their impact on the 

model predictions. These errors and variations are classified 

more generally as uncertainties; where uncertainties are 

classified as either irreducible (aleatory) or reducible 

(epistemic or model error). Examples of aleatory or natural 

variations include variations in geometry from manufacturing, 

material properties, and environmental loads. These types of 

uncertainties can generally be described by a probability 

density function. Epistemic uncertainties are general model 

errors such as limited data to fit a PDF (e.g., few experiments 

to characterize the mean and standard deviation of yield 

strength) and model form (choice of constitutive material 

formulation, boundary conditions, and finite element mesh 

discretization). The uncertainty propagation concept is shown 

in Figure 1. The model parameters are modeled as PDFs on the 

left, an appropriate probabilistic algorithm is used to propagate 

these PDFs through the model (center), resulting in variations 

in the performance of the system shown by the reliability 

function on the right. Many uncertainty propagation methods 

also quantify the importance of the uncertain parameters to the 

overall model prediction uncertainty. These contributions are 

represented by the bar chart on the right. The dashed lines on 

the PDFs on the left represent epistemic uncertainty such as 

limited data used to estimate parameters for the PDF. These 

epistemic uncertainties can also be propagated to quantify their 

impact on the reliability function using appropriate methods.

There are many mature methods to propagating uncertainties 

when they can be modeled by PDFs. These methods are 

usually classified as probabilistic methods and include 

sampling methods, most probable point (MPP) methods, and 

hybrid methods (combined sampling and MPP) among others. 

Problems with ill-behaved limit-states or with multiple limitstates, however, generally require the use of sampling based 

methods or a hybrid approach (Refs. 3 and 4). 

Figure 1.—Uncertainty propagation.

Material Properties

Loadings

Boundary Conditions Location Radius Load S-N Scatter

0

0.2

0.4

0.6

0.8

1

Probabilistic Sensitivity Factor

Life

Reliability

Service

Life

Goal
NASA/TM—2013-217849 3 

Sampling methods such as Monte Carlo or Latin 

Hypercube Simulation (LHS) repeatedly evaluate the 

deterministic model to generate samples of the model 

response, from which the response statistics and probability 

of failure are approximated. A major advantage of sampling 

methods is that the deterministic model does not need to be 

simplified or approximated to perform the probabilistic 

analysis. Thus, problems with multiple failure modes and/or 

ill-behaved response functions can be solved without 

difficulty. Another advantage is that the error in the solution 

can be measured. A significant disadvantage of sampling is 

that the probabilistic analysis can become quite costly due to 

the large number of samples required to compute small failure 

probabilities. One approach to improve the efficiency of 

sampling based methods is to focus the samples in the 

important (failure) regions. This class of sampling methods is 

called importance sampling and can be orders of magnitude 

more efficient than Monte Carlo simulation (Ref. 5). 

Case Study

Cracking of space shuttle main engine (SSME) highpressure oxidizer turbopump (HPOTP) knife-edge seals 

(KES) has been observed in several units. Although this 

cracking did not resulted in the complete fracture of any units, 

the consequences of such a fracture could be severe. As a 

result, life limits had been imposed that effectively required 

the complete replacement of the units following each flight, 

with substantial cost and schedule implications.

Extensive prior investigations by Pratt & Whitney 

Rocketdyne (PWR) and NASA Marshall Space Flight Center 

(MSFC) determined that these seals are sensitive to fluidstructure instability, or flutter. The relationships between 

various build parameters (clearances and gaps) and 

operational environments (local temperatures and pressures) 

and flutter tendencies were established using flutter 

experiments on actual seals. The most severely cracked unit 

was noted to have experienced a flutter event of 

approximately 15-sec duration during an early green run 

(ground test run), although the relative significance of this 

green run versus subsequent flight exposure for the observed 

cracking was not clear. Subsequent rig testing confirmed the 

flutter hypothesis. Structural analysis identified the flutter 

modes most likely to cause cracking and confirmed that local 

flutter stress levels could be severe enough to cause crack 

formation and growth.

To reduce risk of HPOTP recycles impacting the STS 

manifest, effort was undertaken by PWR to develop an on 

pump inspection capability to reduce the recycle schedule 

within the existing Deviation Approval Request (DAR) of 

completing an inspection at 50 percent of the life limit and 

then flying to the life limit. However, the impact of this 

approach on the probability of KES cracking was not known. 

Also unknown were the risk implications of flying the seal to 

the life limit with no inspection as an alternate method to 

reduce schedule risk without significantly increasing technical 

risk.

This effort was performed in parallel to a complete redesign 

of the turbopump knife-edge seal configuration. The goal of 

these predictions was to provide additional information to 

guide decisions on the potential of reusing existing and 

installed units prior to the new design certification.

Problem Statement and Strategy 

This investigation focused on cracking in the Turbine Outlet 

Duct (Outboard) 4 Tooth Seal. Analysis was conducted for 

Tooth A, which exhibited the highest dynamic stresses under 

the relevant flutter modes. “Failure” due to cracking in the seal 

was defined as the development of a 0.075 in. radial through 

crack in the seal tooth, which corresponds to crack growth 

through the entire tooth, up to the edge of the web. This is a 

conservative definition of failure, since cracks have been 

observed to grow into the web without causing complete 

fracture of the seal. The objective of the effort was to assess the 

probability of initiating and growing a crack to this length in 

specific flight units. The analysis accounted for unit-to-unit 

variations in temperature, flutter stress magnitude, and fatigue 

life properties.

The assessment was performed for four different usage and 

inspection scenarios, employing consistent assumptions about 

common variables:

1. Current flight requirements: A new seal experiencing 1 

green run and 1 flight.

2. A new seal experiencing 1 green run and 1 flight, followed 

by an eddy-current inspection, and then an additional 

green run and flight.

3. A new seal experiencing 1 green run and 3 subsequent 

flights with no inspection or replacement.

4. A new seal experiencing 1 green run, 1 nominal flight, and 

1 flight containing an abort.

This investigation focused on the calculation of relative risk 

rather than absolute risk. In other words, how much does the 

risk change for the different usage/inspection scenarios? These 

calculated risk numbers were also put into perspective by 

comparing them with calculated risk numbers for specific units 

with known cracking histories. The number of cracks and crack 

sizes for the units with known cracking histories were used as a 

form of validation data. The models were used to predict the 

variation in crack size for the units with known histories and 

compared to the cracking history to establish credibility in the 

model predictions.

The physics of the fluid-structure interactions are complex in 

this severe and uncertain loading environment, and thus 

empirical models based on experimental data were used for 

portions of the model to predict flutter and flutter stresses. 

Once load conditions were estimated using this combination of 

empirical and numerical models, established fatigue crack 

growth models were applied to predict the extent of crack 
NASA/TM—2013-217849 4 

propagation during green runs or flights. Model predictions 

were compared to selected cracking data to validate the 

model.

Calculations were performed using two NASA-developed 

software tools: NESSUS (Ref. 6) for the probabilistic 

analysis, and NASGRO (Ref. 7) for the fracture mechanics 

analysis. NESSUS is a general-purpose probabilistic analysis 

program with a suite of traditional and advanced probabilistic 

methods. NASGRO, which is the standard fracture control 

software for all NASA Centers, computes fatigue crack 

growth (FCG) and fracture for different materials, loads, and 

structural geometries.

Probabilistic Fracture Assessment Model 

The probabilistic fracture assessment model developed and 

used in this investigation combined a series of submodels 

describing the usage, temperature history, flutter tendencies, 

tooth stresses and numbers of cycles, fatigue cracking, 

nondestructive inspection, and finally the probability of 

failure (Fig. 2). Usage information characterized the different 

sequences of green runs and flights considered for each unit. 

The temperature model predicted the turbine exit temperature

history and associated uncertainty for each unit as a function 

of time. The flutter model used operating conditions and build 

clearances to determine key parameters (ACD Ratio and P)

that are predictors of flutter tendencies and then compared 

these parameters with the flutter limit state to determine the 

flutter condition at each time increment in the history. The 

stress model interpreted the flutter model to predict dynamic 

stress amplitudes in the seal tooth at these time increments and 

assign numbers of cycles based on flutter frequencies. The 

fatigue model used high cycle fatigue and fatigue crack growth 

properties to perform an integrated fatigue crack initiation and 

growth simulation for the seal tooth subjected to these stress 

amplitudes with known stress gradients and superimposed 

static stresses. Where appropriate, the inspection model 

simulated a probabilistic eddy current inspection and 

determined if a crack was detected. The probability of failure 

(crack size > 0.075 in.) occurring in each unit for each usage 

scenario was calculated and compared with benchmark failure 

probabilities calculated for verification units with known usage 

and cracking histories. The analysis accounted for unit-to-unit 

variations and uncertainties in temperature, flutter limit state, 

flutter stress magnitude, and fatigue life properties. 

The probabilistic fracture model components were integrated 

using two NASA-developed software tools: NESSUS for the 

probabilistic analysis and NASGRO for the fracture mechanics 

analysis. The random variables included the temperature 

distribution variations, flutter limit state uncertainty, the 

maximum flutter stress, the flutter frequency, the uncertainty 

on the fatigue crack growth model, and probability of detection 

for the inspection. The number of random variables needed to 

define the temperature model is case specific and also depends 

on the use of measured or model based temperature histories. 

The different cases have different numbers of green runs and 

starts.

Figure 2.—Probabilistic fracture assessment model flow chart.

Mission

Temperature Vs

Time (random)

Actual/Predicted

Temperature Scatter

at 200s (random)

Sequence of Flights

and Green Runs

Uniform Static

Stress

(deterministic)

Dynamic Stress

Gradient

(deterministic)

Tooth Geometry

(deterministic)

Dt increment

(deterministic)

ACD Ratio Vs.

Pressure Model

Flutter Limit

State Model

ACD/DP to Stress Transfer Function

(magnitude,duration)

Flutter Stress

Amplitude

(random)

Total Fatigue Life

(NASGRO)

Probability of

Fracture

Seal Build Clearances

(deterministic)

High Cycle

Fatigue Data

Fatigue Crack

Growth Properties

(random)

Small Crack

Parameter a0

FLUTTER MODEL

STRESS MODEL

FATIGUE MODEL

Inspection

Simulation

Probability of

Detection

Probability of Fracture

With Inspection

INSPECTION MODEL

Measured (Actual)

Temperature Vs

Time (deterministic)

Predicted

Temperature at 200s

(deterministic)

Unit Number

USAGE MODEL

TEMPERATURE MODEL
NASA/TM—2013-217849 5 

The highly nonlinear crack growth response and noncontinuous response functions required the use of a samplingbased method to compute the probability of failure and the 

cumulative distribution function. The probabilistic analysis 

for each unit and usage scenario was performed using the 

Monte Carlo simulation algorithm in NESSUS. 

HPOTP Usage Scenarios

The green run is a start that operates the turbopump through 

all potential power settings to verify correct operation of new 

and rebuilt pumps. Two durations of the green run profile are 

used depending on the rebuild status of the pump: pumps that 

are new or have a major rebuild use the long green run, and 

those with minor rebuilds use a short green run profile. The 

standard flight profile consists of a short duration

104.5 percent power setting followed by the so-called 

“bucket” decrease in power (to reduce aerodynamic forces) 

and then by a long duration 104.5 percent power setting until 

orbit is reached.

Temperature Models

The turbine exit temperature was found to be one of the 

most important variables affecting KES flutter and fatigue 

cracking. This temperature is one of the key inputs to the 

calculation of the ACD ratio and P values (see the following 

section on Flutter Models), which directly impact dynamic 

stresses. Measured turbine exit temperature profiles were 

available for previously flown units. Measured temperatures 

were obviously unavailable for unflown units, so a model was 

developed to estimate the temperatures associated with these 

units. Both the measured and model-based temperatures 

considered uncertainties. These temperature models were then 

combined to generate full temperature histories for each 

usage scenario based on defined power profiles for green 

runs, normal flights, and aborts.

Flutter Models

Evaluation of the flutter rig experimental data by PWR 

indicated that tendencies for flutter were predicted by 

particular combinations of values of the ACD ratio and a 

specific pressure drop P. The ACD ratio is a function of the 

geometry of the seals. The model for flutter developed in this 

study, therefore, consisted of two parts. The first part of the 

model involved predicting values of the ACD ratio and P for 

specific units as a function of build clearances and operating 

conditions (power setting and temperature). The second part 

of the model involved defining a “flutter limit state” (FLS) 

that related these ACD and P values to flutter occurrence. 

Two of the key inputs into the flutter limit state model are 

the definition of the ACD ratio and pressure differential for 

the current unit. The ACD ratio is defined by the effective 

areas of the seal geometry. These values are functions of the 

as-built clearances and associated operating conditions. The 

pressure differential ( P) is measured across the four-tooth 

seal. The ACD and P values were computed using a 

proprietary flow network solver for each unit configuration, 

temperature, and power setting. A simple regression model was 

developed to describe the changes in ACD and P due to 

changes in power level and turbine exit temperature for a 

single unit. This regression model could then be used to predict 

ACD and P values for any unit and any combination of power 

level and turbine exit temperature, given that a single anchor 

point (ACD vs. P for a specific power level and temperature) 

for this unit was provided by a flow network solver.

PWR had developed a flutter test rig to investigate whether 

flow-induced flutter was a credible failure scenario and to 

characterize the flutter sensitivity and dynamic response of the 

KES through parametric variations of flow and clearance 

conditions. These experiments did not include service 

temperatures, two-phase flow, or rotation of an actual unit. 

However, the experiments were expected to predict similar 

flutter strain response to the operating units for comparable 

controlled values of ACD and P, and therefore the flutter rig 

tests were used as the basis for the developed flutter models. 

When flutter occurred, dynamic strains in the seal teeth 

increased dramatically, and therefore large measured dynamic 

strains served as an indication of flutter.

As noted earlier, the ACD ratio and P values were found to 

correlate with tendencies for flutter; in particular, flutter was 

found to occur at higher values of ACD and P. Three flutter 

rig experiments were performed and points in ACD- P space 

were identified corresponding to flutter conditions and noflutter conditions. This line is called the “flutter limit state” 

(FLS).

The validity of this line was verified by comparing predicted 

strains to the experiments. Approximate upper and lower 

bounds on this FLS were set visually to account for some of 

the misclassification of flutter and no-flutter points. These 

approximate upper and lower bounds were assumed to 

correspond to plus or minus three standard deviations from the 

central tendency (mean value) FLS. Uncertainty in the FLS 

was modeled (using these approximate standard deviations) as 

a parallel shift that changed the intercept of the FLS line but 

not its slope. Attempts to refine the FLS uncertainty model 

using more rigorous statistical regressions were not supported

by the available data. The uncertainty model was judged to be 

sufficiently accurate for the relative probability comparisons 

between different units.

Stress Model

Flutter conditions have to be quantified into specific KES 

tooth stress levels in order to perform fatigue assessments. An 

approximate stress model was developed to serve this purpose. 

In this approximate model, the amplitude of the dynamic 

stresses was assumed to be directly proportional to the 

perpendicular distance (d) from the FLS in ACD- P space. At 

the FLS, the dynamic stress amplitude was assumed to be zero. 
NASA/TM—2013-217849 6 

The engine upper operating threshold was used to define the 

maximum possible perpendicular distance (D) away from the 

FLS. The intercept of the engine upper operating threshold 

was defined for a specific unit at 104.5 percent green run 

power setting in ACD- P space.

A wide range of flutter modes is possible in the knife-edge 

seal. The 8th nodal diameter mode (and its close neighbors) is 

believed to be the most severe stress condition and also the 

primary source of cracking of a known cracked unit. 

Therefore, the stress model used in this study was focused on 

the 8th nodal diameter mode.

The maximum possible dynamic stress amplitude value was 

taken to be the predicted stress in the 8th nodal diameter mode 

for the seal at the maximum possible deflection value, which is 

defined by the clearance. This dynamic stress value was 

determined for the known cracked unit and assigned a normal 

distribution with a coefficient of variation of 10 percent. The 

actual standard deviations were not known for this variable, but 

these values were used consistently for all analyses in order to 

support relative probability comparisons between units. A 

uniform static stress was also superimposed, representing 

nominal conditions without flutter.

Fatigue Model

The HPOTP knife edge seals are fabricated from Incoloy 

909 and experience in-service metal temperatures of about 

250 to 350 °F in a two-phase, fuel-rich steam environment 

comprising a mixture of GH2, steam, and liquid water. 

Fatigue crack growth (FCG) rate data for this material in a 

GH2 plus steam environment at 300 °F had been previously 

generated from compact tension (through crack) and surface 

crack specimens under both constant amplitude and Kgradient load histories at a variety of stress ratios. These data, 

corrected for crack closure (Ref. 8), were used to develop the 

NASGRO fatigue crack growth relation. No threshold was 

included in this model.

A small-crack model, originally proposed by El Haddad

(Ref. 9), was used to model fatigue crack growth initiation. 

This small-crack model results in a non-zero growth rate for 

vanishingly small physical crack sizes. As the crack size 

increases, the small-crack effects become increasing 

negligible. In this way, the small-crack model facilitates a 

single FCG analysis that incorporates both initiation and 

propagation phases of life over the entire range of crack sizes. 

This approach is preferable to a more traditional calculation 

of crack “initiation” life based on the failure of a smooth test 

specimen, because the entire KES tooth is actually smaller 

than the customary smooth fatigue specimen. The small-crack 

model was developed from available smooth HCF data at

300 °F in a GH2 + steam environment in both the nominal 

material condition and a pre-exposed (supercharged) material 

condition.

Fatigue cracking was modeled as a corner crack in a 

rectangular plate that transitions into a through edge crack in 

a plate of variable thickness. NASGRO analyses were

performed with a special pre-production version of the 

software in order to take advantage of two new weight function 

(WF) stress intensity factor (SIF) solutions that had not yet 

been implemented in the current (at the time of the 

investigation) production version, NASGRO 5.0. The first new 

SIF solution was a univariant WF solution for the corner crack 

in a plate based on the classical Shen and Glinka approach

(Ref. 10). The second SIF solution was a univariant WF 

solution for the through edge crack in a plate of variable 

thickness with arbitrary stress gradients along the length of the 

plate. Both solutions were modified to include small-crack 

effects. The NASGRO source code was further modified to 

facilitate a direct transition from the corner crack to the 

through edge crack at a crack of 95 percent of the plate width.

Inspection Model

A special eddy current inspection (ECI) method was 

developed for an on pump inspection capability to reduce the 

recycle schedule within the existing Deviation Approval 

Request (DAR) of completing an inspection at 50 percent of 

the life limit and then flying to the life limit. This investigation 

was performed to assess the risk implications of that approach. 

This inspection was included in one usage scenario (denoted as 

Case 2), which contained a green run and flight, followed by 

the ECI inspection, and then (if no crack was detected) another 

green run and flight. The inspection model compared the 

predicted crack length following a green run and flight with a 

detectable flaw size. If the predicted crack length after the first 

green run and flight was less than the detectable flaw size, the 

predicted flaw was not detected. The seal would then be 

subjected to a second green run and flight, from which a final 

crack length would be predicted.

Not all flaws that exceed a nominal detectable length will 

actually be detected. The detectable flaw size is not a 

deterministic value, but has some uncertainty due to the 

efficacy of the inspection method. This uncertainly is 

characterized as the probability of detection (POD). The POD 

is the (cumulative) probability of detecting flaws greater than a 

given crack length.

Probabilistic Fracture Assessment Predictions

The probabilistic fracture assessment predictions were 

performed using (1) “verification” units to demonstrate the 

predictive capability of the developed models and (2) 

“operating” units to predict the probabilities associated with 

current and future flight units. The verification units were 

chosen for their different temperature history severities and 

known cracking conditions. The operating units were recently 

flown or were scheduled for a future flight. The probabilistic 

predictions were performed using the specific temperature and 

build conditions for each unit.
NASA/TM—2013-217849 7 

Verification Units

Verification predictions were first performed for three units 

with known usage and cracking histories. Unit A had 

experienced a severe thermal history and had exhibited severe 

cracking. Unit B had experienced many starts, slightly 

elevated temperatures, and substantial cracking (this unit had 

one through tooth crack of 0.075 in. and numerous smaller 

cracks). Unit C had limited exposure, a benign build, and no 

observed cracking. The fracture assessment model 

successfully predicted the contrasting failure probabilities that 

were consistent with the observed cracking behavior for all 

three units. Unit B is used as the baseline unit for relative 

reliability comparisons of operating units. More informed 

decisions for using a unit for an additional flight without 

inspection can be made by assessing the relative reliability 

with respect to unit B.

The cumulative distribution function (CDF) of the final 

crack size is shown in Figure 3 for each of the three 

verification units. The left y-axis displays the cumulative 

probability in standard normal units. The crack size is plotted 

in log scale along the bottom axis. This CDF indicates 

reliability—the cumulative probability that the crack size is 

less than a specific crack size. The probability of failure is one 

minus the reliability. A comparison of the CDFs in Figure 3 for 

each verification unit demonstrates the validity of the 

developed models. Unit C was expected to have the least crack 

growth due to its location in ACD- P space and benign 

temperatures. Unit A was expected to have the largest cracks. 

Unit B was expected be somewhere in between. The results 

shown in Figure 3 support the relative predictive capability of 

the models. The probabilistic results for the operating units 

summarized in the next section are compared to the model 

predictions for unit B in order to make a relative comparison to 

two known cracked cases.

Operating Units

Using the verified models, predictions were performed for an 

assortment of operating units that had flown recently or that 

were projected for future flights. These computations assessed 

the final crack size distribution and probability of failure for 

the various usage scenarios. All results were normalized 

against similar predictions for the baseline verification unit B.

Figure 3.—CDF of final crack size for the verification units.

Unit A

Unit C Unit B
NASA/TM—2013-217849 8 

Figure 4.—CDF of final crack size for an operating unit.

The deterministic and probabilistic variables needed for the 

probabilistic fracture assessment model were described for 

each part of the model. Each unit analyzed was defined by the 

build parameters and temperature history profiles. The

temperature random variable definitions were different for 

each unit and case (usage scenario). Parameters for the flutter, 

stress, FCG, and inspection models were the same for each 

analyzed unit and usage scenario. The cases are described 

below:

Case 1 requires one green run and one flight. 

Case 2 adds the inspection and another green run and 

flight. Additional random variables were required for 

the temperature model to account for the green run and 

flight after inspection. The inspection model also 

requires the distribution for detectable crack sizes.

Case 3 requires one green run and three flights with 

additional temperature variables to account for the 

temperature variability between each flight.

An operating unit has a known green run temperature 

history, and the initial analysis used the model-based flight 

temperature history for the three operating scenarios. This 

unit was flown during this study so a measured flight 

temperature history also became available. The Case 1 usage 

scenario was also analyzed using the measured flight 

temperature history. The CDFs for this unit for the different 

scenarios are shown in Figure 4 along with the CDF for unit 

B for a relative comparison. Case 1 using the model-based 

temperature history shows a higher reliability compared to 

unit B. Adding two additional flights (Case 3) introduced a 

slight reliability penalty but still demonstrated a higher 

reliability than unit B. The addition of the inspection (Case 2) 

reduced the reliability below that predicted for unit B. The 

additional green run required after the inspection increased the 

exposure at the high power setting so the unit experienced 

additional time in the flutter region. In this case, the additional 

green run required following inspection is more damaging than 

two additional flights. Finally, the unit was also analyzed using 

the actual flown temperature histories. The CDF using the 

measured temperature values demonstrated a higher reliability 

than that using the model-based temperature history. The 

measured flight temperature history removed some of the 

uncertainty from the temperature model. In this case, the 

model-based temperature history was conservative (a lower 

reliability was computed).

Case Study Summary

The impact of different usage and inspection scenarios on 

fatigue cracking due to flutter in SSME HPOTP knife-edge 

seals was assessed. A probabilistic fracture assessment model 

was developed that combined a series of submodels describing 

the usage, temperature history, flutter tendencies, tooth stresses 

and numbers of cycles, fatigue cracking, nondestructive 

inspection, and finally the probability of failure. The 

probability of failure (crack size > 0.075 in.) occurring in each 

unit for each usage scenario was calculated and compared with 

benchmark failure probabilities calculated for verification units 

with known usage and cracking histories. The analysis 

accounted for unit-to-unit variations and uncertainties in 

temperature, flutter limit state, flutter stress magnitude, and 

fatigue life properties. Calculations were performed using two 

Case 1

Case 2

Case 3

Baseline (Unit B)

Case 1 with measured 

temperature
NASA/TM—2013-217849 9 

NASA-developed software tools: NESSUS for the 

probabilistic analysis and NASGRO for the fracture 

mechanics analysis.

Verification predictions were first performed for three units 

with known usage and cracking histories. The probabilistic 

fracture assessment model successfully predicted widely 

contrasting failure probabilities that were consistent with the 

observed cracking behavior for all three units. Numerous 

predictions were performed for an assortment of operating 

units that had flown recently or that were projected for future 

flights, and all of these results were tabulated. These 

computations assessed the final crack size distribution and 

probability of failure for the various usage scenarios. While 

absolute reliability values were not predicted, more informed 

decisions for using a unit for an additional flight without 

inspection can be made by assessing the relative reliability 

with respect to the baseline unit.

Conclusions

This paper describes the development and application of a 

probabilistic fracture assessment model to analyze cracking in 

the SSME HPOTP knife edge seals. The probabilistic fracture 

assessment model consisted of a series of submodels, each of 

which had their own physics, assumptions, and uncertainties. 

The development of each submodel included some level of 

verification and validation leading to improved confidence in 

the predictive capability of the integrated probabilistic 

fracture assessment model. Due to time and cost constraints 

for using these models to make decisions, rigorous model 

V&V was not performed for each submodel. However, many 

concepts for model V&V were exercised during the 

engineering model development process such as verification 

of each submodel, validation against experimental results 

when available, model comparison to historical data for 

model validation, and frequent peer reviews.

Model V&V is a process to establish credibility in model 

predictions. Ideally, carefully controlled validation 

experiments will be designed and performed to validate 

models or submodels. In reality, time and cost constraints 

limit experiments and even model development. Communication of the assumptions and uncertainties are even more 

important due to these constraints. Incorporating UQ into the 

model predictions provides a rigorous approach to 

communicate the impact of assumptions and uncertainties in 

the model predictions. A major benefit of this process is 

communicating the model and data assumptions and 

uncertainties for peer reviews and to the decision makers. Thus 

decisions based on model predictions can be made in light of 

the assumptions and uncertainties. 

References

1. American Institute of Aeronautics and Astronautics, 1998, 

AIAA Guide for the Verification and Validation of 

Computational Fluid Dynamics Simulations, AIAA, 

Reston, VA.

2. American Society of Mechanical Engineers, 2006, Guide 

for Verification and Validation in Computational Solid 

Mechanics, ASME, New York, NY.

3. Thacker, B. H., Riha, D. S., Millwater, H. R., Enright, M. 

P., “Errors and Uncertainties in Probabilistic Engineering 

Analysis,” 2001, Proc. AIAA/ASME/ASCE/AHS/ASC 42nd 

Structures, Structural Dynamics, and Materials (SDM) 

Conference, Seattle, WA, AIAA-2001-1239. 

4. Riha, D. S., Thacker, B. H., Fitch, S. H. K., 2004,

“NESSUS Capabilities for Ill-Behaved Performance 

Functions,” Proc. AIAA/ASME/ASCE/AHS/ASC 45th 

Structures, Structural Dynamics, and Materials (SDM) 

Conf., Palm Springs, CA, AIAA-2004-1832.

5. Wu, Y.T. Computational Method for Efficient Structural 

Reliability and Reliability Sensitivity Analysis. AIAA J. 

1994; 32. 

6. Thacker, B. H., Riha, D. S. Fitch, S. K. Huyse, L. J. and 

Pleming, J. B., “Probabilistic Engineering Analysis Using 

the NESSUS Software,” Structural Safety, Vol. 28, No. 1-

2, 2006, pp. 83-107.

7. Southwest Research Institute (SwRI) and NASA Johnson 

Space Center. NASGRO® Fracture Mechanics and Fatigue 

Crack Growth Analysis Software, Version 5.2 Reference 

Manual, SwRI, 2008.

8. Newman, J. C., Jr., 1984, “A Crack Opening Stress Equation for 

Fatigue Crack Growth,” International Journal of Fracture, Vol. 

24, pp. R131-R135.

9. El Haddad, M. H., Smith, K. N., and Topper, T. H., 1979, 

“Fatigue Crack Propagation of Short Cracks,” Journal of 

Engineering Materials and Technology, Vol. 101, pp. 42-46.

10. Shen G., and Glinka, G., 1991, “Weight Functions for a 

Surface Semi-Elliptical Crack in a Finite Thickness Plate,” 

Theoretical and Applied Fracture Mechanics, Vol. 15, 

1991, pp. 247-255.
REPORT DOCUMENTATION PAGE Form Approved 

OMB No. 0704-0188 

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the 

data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this 

burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. 

Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB 

control number. 

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 

1. REPORT DATE (DD-MM-YYYY)

01-04-2013 

2. REPORT TYPE

Technical Memorandum 

3. DATES COVERED (From - To)

4. TITLE AND SUBTITLE

Model Verification and Validation Concepts for a Probabilistic Fracture Assessment Model to 

Predict Cracking of Knife Edge Seals in the Space Shuttle Main Engine High Pressure 

Oxidizer 

5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S)

Pai, Shantaram, S.; Riha, David, S. 

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

WBS 724297.20.21.12 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration 

John H. Glenn Research Center at Lewis Field 

Cleveland, Ohio 44135-3191 

8. PERFORMING ORGANIZATION

 REPORT NUMBER

E-18636 

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration 

Washington, DC 20546-0001 

10. SPONSORING/MONITOR'S 

 ACRONYM(S)

NASA 

11. SPONSORING/MONITORING 

 REPORT NUMBER

NASA/TM-2013-217849 

12. DISTRIBUTION/AVAILABILITY STATEMENT

Unclassified-Unlimited 

Subject Category: 15 

Available electronically at http://www.sti.nasa.gov 

This publication is available from the NASA Center for AeroSpace Information, 443-757-5802

13. SUPPLEMENTARY NOTES

14. ABSTRACT

Physics-based models are routinely used to predict the performance of engineered systems to make decisions such as when to retire system 

components, how to extend the life of an aging system, or if a new design will be safe or available. Model verification and validation (V&V) 

is a process to establish credibility in model predictions. Ideally, carefully controlled validation experiments will be designed and performed 

to validate models or submodels. In reality, time and cost constraints limit experiments and even model development. This paper describes 

elements of model V&V during the development and application of a probabilistic fracture assessment model to predict cracking in space 

shuttle main engine high-pressure oxidizer turbopump knife-edge seals. The objective of this effort was to assess the probability of initiating 

and growing a crack to a specified failure length in specific flight units for different usage and inspection scenarios. The probabilistic 

fracture assessment model developed in this investigation combined a series of submodels describing the usage, temperature history, flutter 

tendencies, tooth stresses and numbers of cycles, fatigue cracking, nondestructive inspection, and finally the probability of failure. The 

analysis accounted for unit-to-unit variations in temperature, flutter limit state, flutter stress magnitude, and fatigue life properties. The 

investigation focused on the calculation of relative risk rather than absolute risk between the usage scenarios. Verification predictions were 

first performed for three units with known usage and cracking histories to establish credibility in the model predictions. Then, numerous 

predictions were performed for an assortment of operating units that had flown recently or that were projected for future flights. 

Calculations were performed using two NASA-developed software tools: NESSUS® for the probabilistic analysis, and NASGRO for the

fracture mechanics analysis. The goal of these predictions was to provide additional information to guide decisions on the potential of 

reusing existing and installed units prior to the new design certification.

15. SUBJECT TERMS

Seals; Frequency; Knife edge seals; Cracking 

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF

 ABSTRACT

UU 

18. NUMBER

 OF 

 PAGES

16 

19a. NAME OF RESPONSIBLE PERSON

STI Help Desk (email:help@sti.nasa.gov) 

a. REPORT

U 

b. ABSTRACT

U 

c. THIS 

PAGE

U 

19b. TELEPHONE NUMBER (include area code)

443-757-5802 

Standard Form 298 (Rev. 8-98)

Prescribed by ANSI Std. Z39-18