# Presentation: 389432

## Probabilistic Assessment of a Static Frame Model

- Ramesh Rebba, John Cafeo

*Vehicle Development & Research Laboratory*
*General Motors Corporation, Warren, MI*

- Validation Challenge Workshop
- May 21-23, 2006, Albuquerque, NM

## Outline of the talk

- Explain the assumptions made in our analysis
- Go through the four main tasks: calibration, validation, accreditation and prediction
- Address the questions posed in the tasking document for each task
- Discussion over the results
- Questions and comments

## Assumptions

- Beams and bars made of same material
- Material properties are statistically independent from bar-bar or beam-bar
- Random fields to characterize the variation of Elastic modulus along the length of a member
- Non-Gaussian, Stationary assumptions
- Combination of parametric and non-parametric methods

## Assumptions

-   Material follows linear constitutive law
-   Experimental errors and systematic bias are absent
-   Complete fidelity to data
-   Employ only classical statistics
-   Work with single model

## Calibration

## Calibration

- Material model constitutes marginal distribution for elastic modulus *E*(*s*), stationary auto-correlation function *R*(**)

- In our analysis, an exponential correlation function was used

*R*(**) = exp(-*c*|**|)

- where *c* > 0, **  [0, *L*]

*L* is length of a member

- Stationary mean for *E*(*s*) is assumed

## Calibration

- _Marginal Distribution_

- Polynomial chaos expansion used to characterize uncertainty instead of standard distributions

- where ** ~ *N*(0, 1)

- 0() = 1            0() = **          2(**) = **2 – 1
- 3(**) = **3 - 3**    4(**) = **4 – 6**2 + 4 ....

- Johnson SB distribution offers a flexible alternative way

## Calibration

- _Marginal Distribution_

- P. C coefficients calculated using regression – requires empirical CDF of material data

- Given a set of ordered observations ***x*** = {*x**1*, *x**2*, *x**3*, ..., *x**n*}

- Empirical CDF

-  or *F**X*-1(*p*) = ***x ***[int(*np* + 0.5)]

- For every *x**j*, there is a corresponding variable ***j* in standard normal space such that ***j* = -1[*F**X*(*x**j*)] and hence *i*(***j*) can be calculated

## Calibration

- _Marginal Distribution_

- Using *x**j* and *i*(***j*) for *j* = 1 to *n*, a regression analysis can be set up to estimated the coefficients *a**i*

| j | x |  = -1[(j - 0.5)/n]  | 1() | 2() | 3() |
| --- | --- | --- | --- | --- | --- |
| 1 | x1 | 1 | 1 | 12 – 1 | 13 - 31 |
| 2
.
. | x2
.
. | 2
.
. | 2
.
. | 22 – 1
.
. | 23 - 32
.
. |

- An alternative simulation method tends to be inaccurate for estimating higher-order coefficients

## Calibration

- _Random field Simulation_

- Gaussian field **(*s*) with zero mean and autocorrelation *R*(**) is simulated first, and subsequently transformed into Non-Gaussian space

*E*(*s*) = *a**0* + *a**1 ***(*s*) + *a**2* (**(*s*)2 - 1) + *a**3* (**(*s*)3 – 3**(*s*)) + ....+ **

- Data on material properties is used to calculate *a**0*, *a**1*, ...

- Data on elongation is used to calibrate correlation parameter *c* for the underlying Gaussian field
- Verify that resulting covariance for *E*(*s*) is positive definite

## Calibration

- _Simulation of Gaussian field_

-  Spectral method
-  Sampling theorem
-  Karhunen-Loeve

-  - Treat it as a problem of simulating correlated variables

- Cholesky decomposition is computationally superior to K-L expansion especially when *E*(*s*) is generated at discrete locations in the range [0, *L*]

-  - Issues with continuous KL expansion

## Calibration

- _Simulating bar elongation_

-  For a particular value of *c*, generate correlated standard normal variables using *R*(**) at *M* locations along the length of the bar

-  Translate those variable into *E*(*s**i*) for *i* = 1 to *M* using PC expansion
-  Compute the following integral numerically

-  Repeat above process to generate samples of ***L*

*L* = 0.2

## Calibration

- The goal is to estimate *c* such that the distributions of elongations from simulation and given data match

- The ‘distance’ between two CDFs is measured using Anderson-Darling Statistic

- Find c  Min:

- If model output follows CDF *F**M* and data ***x***** **is given,

## Calibration

-  Likelihood methods of estimating parameters may fail in tail regions
-  Deriving CDF of simulated output is much easier and straightforward than PDF

-  The correlation parameter ‘*c’* controls the variance of ***L*
* *If the mean value of ***L* does not match with that of data, then a  constant perturbation term ***E* will be added to *E*(*s*)
-  P. C coefficients are not updated and the discrepancy in CDF’s arises only due to our covariance model selection and stationary random field assumptions

## Calibration

- _Results_

- Polynomial chaos coefficients and correlation parameter *c* and perturbation term ***E* estimated

| Nc | a0 | a1 | a2 | a3 |  | c | E |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 5 | 12.36 | 1.624 | 0.789 | - | 0.589 | 180 | -0.7 |
| 20 | 11.611 | 0.920 | 0.373 | 0.116 | 0.173 | 24 | -0.225 |
| 30 | 11.643 | 0.857 | 0.294 | 0.057 | 0.13 | 23.5 | -0.275 |

- Higher-order coefficients have t-ratios smaller than 1.0 and hence dropped from regression, also avoid over fitting distributions to smaller data sets

## Calibration

- _Plots_

- Make sure empirical CDF of elastic modulus ‘*E*’ matches with the fitted distribution from P.C

- Nc=20

- Nc=30

- Nc=5

## Calibration

- _Plots_

- Make sure empirical CDF of the observed elongation matches with that from the simulation

- Nc=20

- Nc=30

- Nc=5

## Calibration

- How is the variability in model parameters associated with material models characterized and estimated using data?
- 2. What is the benefit of additional data, as provided by the increased data in later sets, on the material model characterization and calibration? How is this benefit/impact quantified?
- 3. How many model evaluations were required?

## Validation

- _Validation metric_

- The choice of metric depends on the type of data and model output
- In this problem, we like to test the null hypothesis that the observed sample set belongs to the population of simulated elongations

- Anderson-Darling Test Statistic

- Reject null if *A*2 > *A*2crit, 0.05

## Validation

- _Results_

- Model output simulated for a bar length different from the one used in calibration

*L* = 0.8

| Model output |  |  | Test data |  |  |
| --- | --- | --- | --- | --- | --- |
| Nc | Mean | Stdev | Nv | Mean | Stdev |
| 5 | 0.211 | 0.00329 | 2 | 0.2035 | 0.003535 |
| 20 | 0.2124 | 0.00497 | 4 | 0.204 | 0.003559 |
| 30 | 0.2125 | 0.00471 | 10 | 0.2044 | 0.004088 |

- Based on 10,000 simulations, statistics of ***L* obtained

## Validation

- _Results_

| Nc | Nv | A2 | A2crit, 0.05 | Decision on Null |
| --- | --- | --- | --- | --- |
| 5 | 2 | 5.52 | 0.388 | Reject |
| 20 | 4 | 6.49 | 0.566 | Reject |
| 30 | 10 | 15.02 | 0.685 | Reject |

- Samples followed nearly Normal distribution, A-D test statistic adjusted

## Validation

- _Results_

| Nc | Nv | c | E |
| --- | --- | --- | --- |
| 5 | 2 | 130 | -0.285 |
| 20 | 4 | 42 | 0.238 |
| 30 | 10 | 28 | 0.168 |

- The discrepancy in the distributions from simulation and test data at *L* = 0.8 could be due to extrapolation of the material model beyond *L* = 0.2 at which it was calibrated

- Thus for all bars of length L  [0, 0.8], the material model can further be updated/ calibrated using the ensemble validation data

## Validation

- _Model error_

- The discrepancy between model output and observation characterized as model form error or a bias

- Distribution for model form error represents possible bias for predictions of bar lengths L  [0.2, 0.8]

*b* (*s*) = *E*(*s*, *c**v*, **Ev) – *E*(*s*, *c*, ***E*)

- Model bias is a random variable. If accuracy threshold ***b* is chosen, model reliability can be computed as

*r* = *P*(-***b* < *b*(*s*) <***b*)

## Validation

- _Other metrics_

-  Effect of sample size on metric
-  t-test, chi-square test
-  Bayesian hypothesis testing
-  Model reliability metric
-  Probability intervals
-  Other goodness-of-fit (GoF) tests

## Validation

-  Validation metric
- Model accuracy
- Model adequacy
- Recommended model usage
- Value of more tests
- Function evaluations

## Accreditation

- Each bar is made of same material but statistical properties are independent

- First simulate three different bars (1, 2, 4) by generating three independent random fields

- Bar 3 does not affect the displacement at midpoint of beam

- Calculate the displacement using given formulae
- Generate distribution of displacement for each case by repeatedly simulating 3 sets of random bars

## Accreditation

- _Results_

**x**

**x**

## Accreditation

- _Results_

| Nc | Nv | Na | A2 | A2crit, 0.05 | Decision on Null |
| --- | --- | --- | --- | --- | --- |
| 5 | 2 | 1 | 0.871 | 2.73 | Accept |
| 20 | 4 | 1 | 0.695 | 2.73 | Accept |
| 30 | 10 | 2 | 2.086 | 2.61 | Accept |

| Case | Na | C | E |
| --- | --- | --- | --- |
| 1 | 1 | 130 | -0.475 |
| 2 | 1 | 42 | 0.075 |
| 3 | 2 | 23 | -0.16 |

- The material model is further updated/ calibrated to include the accreditation data.

## Accreditation

-  Validation metric
- Model accuracy
- Model adequacy
- Recommended model usage
- Value of more tests
- Function evaluations

## Prediction

- Simulate four different bars by generating four independent random fields and corresponding elongations

- Calculate the displacement using given formulae
- Use appropriate moment functions
- Generate distribution of displacement for each case by repeatedly simulating random bars
- Compute the failure probability by counting the number of times the displacement exceeded 3 mm and divide it by total runs

## Prediction

| Val. | Acc. | c | E | Mean | Stdev | Pf |
| --- | --- | --- | --- | --- | --- | --- |
| Y | Y | 130 | -0.475 | -2.742 | 0.0329 | 2.39e-15 |
| Y | N  | 130 | -0.285 | -2.697 | 0.0314 | 2.61e-22 |
| N | N | 180 | -0.7 | -2.798 | 0.0313 | 5.2e-11 |

| Val. | Acc. | c | E | Mean | Stdev | Pf |
| --- | --- | --- | --- | --- | --- | --- |
| Y | Y | 42 | 0.075 | -2.742 | 0.0298 | 2.99e-18 |
| Y | N  | 42 | 0.238 | -2.703 | 0.0291 | 1.28e-024 |
| N | N | 24 | -0.225 | -2.815 | 0.041 | 3.73e-06 |

| Val. | Acc. | c | E | Mean | Stdev | Pf |
| --- | --- | --- | --- | --- | --- | --- |
| Y | Y | 23 | -0.16 | -2.787 | 0.0392 | 2.88e-08 |
| Y | N  | 28 | 0.168 | -2.708 | 0.0337 | 2.98e-18 |
| N | N | 23.5 | -0.275 | -2.816 | 0.039 | 1.32e-06 |

*N**c* = 5 *N**v* = 2 *N**a* = 1

*N**c* = 20 *N**v* = 4 *N**a* = 1

*N**c* = 30 *N**v* = 10 *N**a* = 2

## Prediction

- Sources of error in reliability prediction
- 	- Model form error
-              - Inadequate data at calibration, validation, accreditation
-              - Reliability analysis technique
-              -  Extrapolation of model beyond validation domain
-              - Questionable experimental precision

- Uncertainty due to each source must be identified and propagated through the reliability prediction model

- Confidence intervals for reliability estimation could be determined

## Prediction

- How are the variabilities associated with previous steps incorporated in model prediction?
- What is the model prediction of probability of failure?
- How does one assess confidence in the resulting prediction of probability?

## Discussion

- The challenge problem is realistic since we have limited data only in most practical situations
- The problem is just one simple case among various cases encountered in automotive industry
- Possibility of errors in development and solution of the challenge problem
- Different research organizations developing validation procedures
- Terminology to implementation

## Thank You