# Presentation: 831372

## NASA Langley’s Approach towards a Solution of the Sandia’s Structural Dynamics Challenge Problem

**L.G. Horta, S.P. Kenny, K.B. Elliott, K.B. Lim,**
**NASA Langley Research Center**
**L. Crespo,**
**National Institute of Aerospace**
**and J-S. Lew**
**Tennessee State University**

## Sandia’s Challenge Systems

**The challenge was formulated around three systems:**
**• ****A ****subsystem**** that isolates phenomena deemed important by problem specialists, albeit in a configuration that is not fully-relevant to the target application.**
**• ****An ****accreditation**** system that is more complex, and more relevant to the target application; it is a configuration that is more expensive to prototype and to test.**
**• ****A ****target**** application, which provides both a decision context and an accompanying regulatory requirement, and serves as the basis for the study**.

**Subsystem**

**Accreditation**

**Target**

## Sandia Workshop Challenge

- Objective:  To develop a “simulation based” approach to assure that sensitive components of a substructure will survive a prescribed shock profile when integrated into the target structure. Specifically, the acceleration of mass 3 in the subsystem needs to satisfy

**Regulatory Requirement**

## Models Not to be Confused

- There are two types of models
- 	1- LTI linear time invariant physics model
- 3-DOF LTI
- Accreditation system
- Target system
- 2- Probabilistic model (PM) of LTI parameter uncertainties

## Questions to be addressed 

- Q1. 3-DOF LTI structure adequacy: Is a Linear Time Invariant (LTI) representation of the 3-DOF subsystem adequate to describe  the nonlinear response  data? (i.e. do we have the adequate physics?)
- Q2. Comparison Metrics: What metrics are used to measure the agreement between the model* predictions and the measurements? (e.g. Probabilistic moments and confidence intervals of maximum acceleration, RMS accel,  modal parameters, principal values...)
- Q3. Model Accuracy: How was the metric estimated, what is the computed value, and how confident are you in the numerical results?
- Q4. Model Adequacy: What is the acceptance/rejection criteria (i.e., value of the comparison metric) and is the model found to be adequate by these criteria? Specifically, is the structure of the JPDF adequate and physically meaningful? (no negative damping, or mass, or stiffness, or...)

- *  Model refers to LTI + JPDF of LTI parameters

## Questions to be addressed(Cont’d)

- Q5. Recommended Model Usage: How do you recommend that the model obtained using the calibration data be used for the validation data assessment? Is updating of the calibration model recommended before using it to evaluate the accreditation data? Describe the updating process if required.
- Q6. Value of More Tests: What is the impact on accuracy assessments when increasing the number of tests in the ensemble database. How is this benefit/impact quantified?
- Q7. Function Evaluations: How many function evaluations were required to do the problem? (Calibration, Validation, accreditation, and target.)

## Some Acronyms

- CI confidence interval
- CDF Cumulative Distribution Function
- DOF degrees-of-freedom
- FORM first order reliability methods
- JPDF Joint Probability Distribution Function
- LTI Linear Time Invariant
- MLE Maximum Likelihood Estimate
- SV singular values

## Is the LTI representation of the 3-DOF subsystem adequate to describe the nonlinear response data? 

- To assess adequacy of the 3DOF LTI, two metrics are used

- How are the metrics computed?
-  Metric 1: Compute max/min singular values of the transfer function at each of the spectral lines and compare to test.
- Metric 2: Given the parameters and disturbances input the maximum acceleration, the mean of the maximum acceleration, and the sampled standard deviation of the maximum acceleration are used to compute the 99.9 % CI using a t-distribution*. *

**Metric 1: Transfer function Max/Min Singular Values (SV) over all system realizations**

**Metric 2: 99.9 % CI using t-dist of output maximum acceleration and/or MLE**

## Subsystems for Calibration and Validation

- Random
- (Low,
- Med,
- High)

- Calibration

- Validation

- Shock
- (Low,
- Med,
- High)

**20 Statistically **
**Independent **
**Realizations**

**20 Statistically **
**Independent **
**Realizations**

## Comparison of Singular Values (Metric 1) Using Sandia Script for 3-DOF LTI

**Calibration Data**

**Validation Data**

**3-DOF LTI Script Unacceptable**

## Comparison of Maximum Acceleration  Metric 2 Using Sandia Script for 3-DOF LTI

- -Max Acceleration prediction errors < 33 %,
- -Mean of Max Acceleration prediction error < 9%
- -3-DOF LTI accel results within 99.9% CI

**Calibration Data**

**Validation Data**

## Q1: Since the 3-DOF LTI system seems acceptable in term of maximum acceleration CI but unacceptable from the SV metric.  A revised NASA script was implemented for the 3-DOF LTI.   (no uncertainty)

## Comparison of Singular Value Metric 1 Using NASA Script for 3-DOF LTI

**Calibration Data**

**Validation Data**

**3-DOF LTI Script acceptable**

## Comparison of Maximum Acceleration  Metric 2 Using NASA Script for 3-DOF LTI

- -Max Acceleration Peak error < 18%,
- -Mean of max Acceleration prediction Error < 3%
- -3-DOF LTI accel within 99.9% CI

**Calibration Data**

**Validation Data**

## Probabilistic Modeling (PM) of Parameter Uncertainties

## Probabilistic Model (PM) Development Using Parameter Population

**F(p)**

**A******

**B**

**C**

***p******1***

***p******2***

***p******3***

**T****-1**

***n******1***

***n******2***

***n******3***

**Matching 1****st**** & 2****nd**** Moments via Cholesky Factorization**
**And MLE**

**Matching**
**1****st****,2****nd**** 3****rd**** Moments, and CI for finite sample via nonlinear optimization and MLE**

****

**Parameter Space**

**Design **
**Space**

**P**

****

**n x n**

**n x n**

**n x m**

**n x m**

**z**

**y**

**LTI**
**Model**
**structure**

**Orthonormal**
**Decomp.**

**m x m**

**Orthonormal**
**Decomp.**

**Strictly**
**positive**

**Select distribution**

**Select distribution &**
**Optimized dist. **
**parameters**

**m x m**

***i***

****

***x***

***x******e***

******e**

**PM-1**

**PM-2**

**Select basis functions**

**Kernel Density bandwidth (variance) computed to minimize Mean Integrated Square Error**

***y******e***

**PM-3**

## PM-1 Steps to Probabilistic Modeling of Parameter Variations (Approach 1) 

## PM-1 99.9 % MLE CI Using 

- PM-1 99.9 % MLE CI Using
- Calibration Data

**Covariance in Normalized Space from Exp. Data**

**Computed Covariance for New Population (20 Param. Vectors)**

**Number of experiments**

## PM-1 99.9 % MLE CI Using 

- PM-1 99.9 % MLE CI Using
- Calibration and Validation Data

**Covariance in Normalized Space from Exp. Data**

**Computed Covariance for New Population (40 Param. Vectors)**

**Number of Experiments**

## PM-1 Comparison of Singular Value Metric 1 (Population size 200)

**Calibration Data**

**Calibration and Validation Data**

## PM-2 Steps to Probabilistic Modeling of Parameter Variations (Approach 2) 

- Decompose and truncate the normalized parameter observation matrix *P *by neglecting small singular values: *P* = *USV*  *ULv* = *Av*
- This leads to the transformation *z* = A*y, *where *z* is an *n* x 1 vector of uncertain parameters, A is an *n* x *m* matrix and *y* is a *m* x 1 vector.
- From the *n* equations available, extract *m* such that *x *= B*y,* where B is non-singular and *x* is a *m* x 1 vector extracted from *z*. This leads to* z* = AB-1*x*.
- Let *x = Ci, *where* C* is a *m* x *m* matrix and* i* is a *m* x 1 vector of independent random variables. *C* and *i* are to be determined. *C* is used to correlate the variables in *z*. Let each component of *i* to have a prescribed structure, e.g. lognormal. The required JPDF is fully prescribed by *C*, and the parameters of the PDFs of the components of *i*. Denote these parameters as *q, *i.e*. i*(*q*).
- The JPDF of *x* is sized by
- 				<*C*,*q*> = *argmin *{*e*T*We*}  subject to
- 		 	                         *AB*-1*C > 0,  *(*i*)  +
- 	where *e *measures the “offset” between the experimental observations and the ones corresponding to the model, and *W* is an arbitrary, strictly positive, weighting matrix. By “offset” we used the 1st, 2nd, and 3rd order moments of the *m* parameters in *x* as well as their MLE-based CIs. For the model, the    moments were calculated analytically and the CIs were calculated using      HSS and a normal SPDF. Constraints are used to prevent unfeasible plants.

## PM-2 Results from Matching Moments 

- for Calibration Data

- (-•- data,
- -o- model)

## PM-2 CI Results from Matching Moments Only 

**for Calibration data**

- (-•- data,
- -o- model)

## PM-2 Results from Matching Moments 

- and CI for Calibration Data

- (-•- data,
- -o- model)

## PM-2 CI Results from Matching Moments & CIs 

**for Calibration data**

- (-•- observed,
- -o- predicted)

## PM-2 Comparison of Singular Value Metric 1 (Population size 200)

- Calibration Data

- Calibration & Validation

## PM-3 Steps to Probabilistic Modeling of Parameter Variations (Approach 3) 

## PM-3 Results from Matching MeanUsing Calibration and Validation Data

**Normalized Mean**

## PM-3 Results from Matching Covariance UsingCalibration and Validation Data

## PM-3 Comparison of Singular Value Metric 1 (Population size 200)

- Calibration Data

- Calibration & Validation

## Accreditation System Results

## Verification of Accreditation System Response Bounds PM-1 (Population Size 200)

**Circles ******** 99.9% CI using t-distribution **

## Verification of Accreditation System Response Bounds PM-2 (Population Size 200)

**Circles ******** 99.9% CI using t-distribution **

## Verification of Accreditation System Response Bounds PM-3 (Population Size 200)

**Circles ******** 99.9% CI using t-distribution **

## Target System Results

## Target System Mass 3 Acceleration CDF using Calibration Data (Population Size 5000)

**Pf=0.17**

**Pf=0.22**

**PM-1**

**PM-2**

**Pf=0.20**

**PM-3**

**Probability of mass 3 on target system to exceed **
**the 18000 in/sec****2**** regulatory requirement **

## Target System Mass 3 Acceleration CDF using Calibration and Validation Data (Population Size 5000)

**PM-1**

**PM-3**

**Pf=0.20**

**Pf=0.13**

**Pf=0.12**

**PM-2**

**Probability of mass 3 on target system to exceed **
**the 18000 in/sec****2**** regulatory requirement **

## Other Approaches that were tried!!!

- UQ using robust control tools could not recover PM models properly (K.B. Lim)
- Interval modeling provides insight into physical parameter dependencies, uncertainty interval sizes with finite data sets, sensitivity, and maximum acceleration intervals but it cannot not provide probabilistic models to extrapolate parameter sets outside given data. (J-S Lew)
- Moving Least Square Response Surface Technique when use to compute probabilities using FORM, requires adaptive meshing (L.G. Horta)
- FORM functions showed significant errors for high values of probabilities (S.P. Kenny, L. Crespo, L. Horta)

## LaRC Internal Benchmark Problem 

- _Benefits_
- Provides a “truth model” for probabilistic modeling techniques
- Facilitates study of the impact of limited data sets
- Allows us to discriminate between different modeling methods

**LaRC Internal Benchmark Problem**** **

- _Model attributes_
- Nonlinear Subsystem model created in Matlab Simulink
- Multivariate (fully correlated 3x3) lognormal variables used to describe mass and stiffness uncertainties.
- Nonlinear cubic stiffness (softening spring) connected mass one to ground described by independent normal distribution.
- Nonlinear cubic damping connected mass one to ground described by independent normal distribution.
- C=*K+*M proportional damping model
- Parameter values chosen to “closely” match the Sandia problem
- Scripts written to generate linear model and “Psi” vector for consistency with Sandia’s philosophy

**M****3**

**M****2**

**M****1**

**K****1**

**K****2**

**K****3**

**C****2**

**C****3**

**Weakly**
**Nonlinear**
**Connection**

## LaRC Internal Benchmark Problem (Lessons Learned)

**LaRC Internal Benchmark Problem ****(Lessons Learned)**

**Nonlinear uncertain damping model resulted in a very challenging problem due to the requirement to work within a LTI framework ****(linearized damping models does not adequately capture physics****) – Got to capture damping well.**
**Nonlinear modeling may dramatically improve uncertainty models (reduction in dimensionality) and provides a framework too discriminate between nonlinearities and variabilities.**

**Linear Truth Model**

**Kernel Density Model**

**Pf=0.1326**

**Pf=0.1230**

**Comparison Using a linear **
**uncertain representation**

## Concluding Remarks

**Three methodologies for uncertainty modeling were derived**
**CDFs of performance metric of interest for three methods were very similar.**
**Target system regulatory requirement shown to be in violation by all three methods with probability ranging from 0.12 to 0.22**
**Probabilistic model development methods matched 1****st****, 2****nd****, 3****rd**** moments, confidence intervals for a finite number of observations**
**PMs derived using calibration data only shown to be adequate in terms of both metrics and probability assessment   **
**Physical parameter uncertainty modeling allowed for probabilistic models with significant reductions in the number of required independent parameters, from 15 to 4 in PM-2 and from 15 to 5 in PM-3, to fully described the JPDF**
**Time and frequency domain metrics are extremely valuable to gain insight into model adequacy (with and w/o external inputs influence)**

## BackUp Charts

## Did we answer the Questions??

- Q1. 3-DOF LTI structure adequacy: Is a Linear Time Invariant (LTI) representation of the 3-DOF subsystem adequate to describe  the nonlinear response  data? (i.e. do we have the adequate physics?)
- Yes, in terms of singular values and confidence intervals
- Q2. Comparison Metrics: What metrics are used to measure the agreement between the model* predictions and the measurements? (e.g. Probabilistic moments and confidence intervals of maximum acceleration, RMS accel,  modal parameters, principal values...)
- Stated for each method in the presentation
- Q3. Model Accuracy: How was the metric estimated, what is the computed value, and how confident are you in the numerical results?
- See method development section
- Q4. Model Adequacy: What is the acceptance/rejection criteria (i.e., value of the comparison metric) and is the model found to be adequate by these criteria? Specifically, is the structure of the JPDF adequate and physically meaningful? (no negative damping, or mass, or stiffness, or...)
- Stated in presentation

## Questions to be addressed(Cont’d)

- Q5. Recommended Model Usage: How do you recommend that the model obtained using the calibration data be used for the validation data assessment? Is updating of the calibration model recommended before using it to evaluate the accreditation data? Describe the updating process if required.
- Stated in presentation
- Q6. Value of More Tests: What is the impact on accuracy assessments when increasing the number of tests in the ensemble database. How is this benefit/impact quantified?
- No numerical results at hand to answer this question.  Planned to use NASA benchmark problem for this purpose
- Q7. Function Evaluations: How many function evaluations were required to do the problem? (Calibration, Validation, accreditation, and target.)
- Indirectly answered but methods efficiency not at the top of the list of the important issues considered

## Estimated Number of Monte Carlo Runs Required to Achieve a Prescribed Probability Accuracy with 95% Confidence

**5000 runs**

**10% accuracy**

**5 % accuracy**

**1 % accuracy**