# Presentation: 101999

## Thermal Validation Challenge Problem

- Kevin Dowding and Martin Pilch
- Validation and Uncertainty Quantification, 1533
- Sandia National Laboratories
- Richard Hills
- Dept of Mechanical Engineering
- New Mexico State University

**Validation Challenge Workshop**
**Albuquerque, MM**
**May 22-23, 2006**

- Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy under contract DE-AC04-94AL85000.

## High-Consequence Regulatory Decision For an Intended Application

**Safety critical devices that demonstrate unit-to-unit variability**
**The intended application of the devices are exposed to environments that are well characterized, *****q=3500 W/m******2***** and thickness *****L=1.90 cm*****, neither having uncertainty**
**Regulatory Requirement**

- -k dT/dx = *q*

*T**s*

- x

- dT/dx = 0

*L*

***Participants are asked to 1)assess whether the device meets ***
***the regulatory requirement using the model and 2)quantify ***
***their confidence in the assessment.***

## Model to assess is (linear) transient heat conduction

**Ground rules:**
**Model form is fixed**
**All inputs precisely known except (***k***, *****C**P***)**
**Thermal properties (***k***, *****C**P***), and possibly other parameters, can be “adjusted”  **
**Same model form is used in the hierarchy of validation exps **
**Magnitudes of heat flux (***q***) and thickness (***L***) are different in the various validation exps**

**Model:**

## Overview of theHierarchal Validation Experiments

**Material Characterization (not shown Fig)**
**Estimate (measure) values for ***k*** and *****C**P ***over a range of temperatures**
**Ensemble Validation**
**Explore a range of (***q,L)*** combinations (configurations)**
**Sensitivity to ***L*** and ability to extrapolate on ***q ***are studied**
**Limited replicates are available**
**Accreditation **
**More relevant to application, but outside previous database of exps (ensemble)**
**Very limited number of exps (usually a single exp)**

## Study sensitivity to the number of exps at each level of the hierarchy

**Different number of exps at each level in the hierarchy **
**Low, Medium, and High number of exps**
**Goal is to understand how analyses (and conclusions) are impacted by the number of exps**
**Look at comparable number of exps for each level in the hierarchy**

| Exp runs | Material Characterization (Nc) | Ensemble Validation (Nv) | Accreditation (Na) |
| --- | --- | --- | --- |
| Low | 6 | 1 per Config./ 4 total  | 1 |
| Medium | 20 | 2 per Config./ 8 total | 1 |
| High | 30 | 4 per Config./ 16 total | 2 |

## Material Characterization

## Material Characterization

**Estimate (measure) values for ***k*** and *****C**P ***over a range of temperatures and samples**
**Did not involve model considered in validation **
**Nominal dimensions of specimens tested are representative of the intended application **
**Manufactured with the same processes used in the intended application**
**Reasonable assumptions:**
**diagnostic variability or uncertainty is negligible compared to specimen-to-specimen variability**
**estimates of thermal conductivity and volumetric heat capacity are independent**
**estimates for a given property at different temperatures are independent (i.e., from characterizing a different sample). **

## Material Characterization data(Nc = 20)

## Ensemble Validation Experiments

## Ensemble validation conducted over four configurations

**Exps conducted at lower heat flux values (***q***), and values of ***L ***that bound that specified in the regulatory requirement **
**Each exp conducted on a different (randomly selected) sample **
**Replicates at each configuration (for medium/high levels of exp data)**

| Exp
Configuration | Heat Flux, q
(W/m2) | Thickness, L
(cm) |
| --- | --- | --- |
| 1 | 1000 | 1.27 |
| 2 | 1000 | 2.54 |
| 3 | 2000 | 1.27 |
| 4 | 2000 | 2.54 |

## Ensemble Validation Data(Nv=8, 2 exps/configuration)

**Ensemble Validation Data****(***N**v***=8, 2 exps/configuration)**

- Config 1
- (1000W/m2,1.27cm)

- Config 2
- (1000W/m2,2.54cm)

- Config 3
- (2000W/m2,1.27cm)

- Config 4
- (2000W/m2,2.54cm)

## Accreditation Validation Experiment

## Accreditation Experiment 

**Conditions more relevant to the regulatory requirement, still at a lower heat flux  **
**Each experiment conducted on a different sample (when applicable)**
**Measure temperatures at ***x=0*** and ***x=L/2*

| Exp
Configuration | Heat Flux, q
(W/m2) | Thickness, L
(cm) |
| --- | --- | --- |
| 5 | 3000 | 1.90 |

## Accreditation Experiment(Na=1)

**Accreditation Experiment****(***N**a***=1)**

## Summary

**Asking participants to assess the *****model accuracy***** and ****model adequacy**** at each level in the validation hierarchy**
**Model accuracy/adequacy defined by the participants**
**Model updating (calibration) is permitted**
**(Option to) use all the information available to assess the ****regulatory requirement**** and the ****confidence**** in the assessment**
**More interested in understanding the assessment methodology and relating validation at various levels (hierarchy)  than the outcome for this problem**

## Thermal Problem Speakers

| 9:00 am to 9:45 am | Model Validation Metrics for the Thermal Challenge Problem
Sankaran Mahadevan, J. McFarland, and X. Jiang Vanderbilt University |
| --- | --- |
| 9:45 am to 10:30 am | Model Validation, Calibration and Predictive Capability
Scott Ferson, Applied Biomathematics
William L. Oberkampf, Sandia National Laboratories |
| 10:30 am to 11:00 am | Break |
| 11:00 am to 11:45 am | A Bayesian Analysis of the Thermal Challenge Problem 
F. Liu and J. Berger, Duke University 
M. J. Bayarri, Valencia 
R. Paulo, Bristol
J. Sacks, National Institute of Statistical Sciences (NISS)  |
| 11:45 am to 12:30 pm | Another Bayesian Analysis of the Thermal Challenge Problem
Dave Higdon, and Charlie Nakhleh Los Alamos National Laboratory |
| 12:30 pm to 1:45 pm | Lunch (provided), Alvarado F |
| 1:45 pm to 2:30 pm  | Thermal Problem Discussion  |

## Topics

**What is the role of calibration in validation??**
**How should a bias term (kludge, patch discrepancy) be incorporated into the model? **
**What metrics should we use? **
**Hypothesis test**
**Discrepancy model**
**How do we formulate the model adequacy? **

## Structural Dynamics-Driven Topics

**Language:**
**Calibration/Validation**
**Identification of analysis approach all relevant models**
**Physical processes**
**Uncertainty models**
**Validation criteria (e.g. relevance to reg req’t)**
**Validation metrics**
**Communication**