Quantification of the uncertainty of NDE

Department of Quantum Science and Energy Engineering,
Graduate School of Engineering, Tohoku University

Noritaka Yusa

Contents

1. Background & motivation
2. Uncertainty of NDE(1) Flaw detection(2) Flaw evaluation

3. Summary
2023/10/10

Risk–based Management of Energy Infrastructure2

1．Background and motivation

We need to accept:

 Limited resource Not parts but a whole No 100% safety

Risk-based management
2023/10/10

Risk–based Management of Energy Infrastructure3

1．Background and motivation
PRA/PFM
RBI
Prediction/
Planning

Repair/Replacement????
Nondestructive
Inspection
Inspection

Evaluation????

PFM?

Fig. General flow of maintenance activity
2023/10/10

Risk–based Management of Energy Infrastructure51．Background and motivation
The role of nondestructive inspection

detection & evaluationof a flaw
to estimate the size ofa detected flaw
to confirm whether
there is a flaw or not

2023/10/10

Risk–based Management of Energy Infrastructure

7

1．Background and motivation
To account for the uncertainty of NDT

detection & evaluation“safety factor”

 true size = estimation±a“minimum detectable flaw size”

 a flaw larger than a certain size->detectable a flaw smaller than a certain size->undetectable

Incompatible with “Risk.”
2023/10/10

Risk–based Management of Energy Infrastructure11

1．Background and motivation
PRA/PFM
RBI
Prediction/
Planning

Repair/Replacement????
Nondestructive
Inspection
Inspection

Evaluation????

PFM?

Fig. General flow of maintenance activity
2023/10/10

Risk–based Management of Energy Infrastructure

12

1．Background and motivation
It is probable that risk-based management ofinfrastructure would be significantly enhanced, if

the uncertainty of nondestructive
evaluation is well quantified,
but the number of such studies is quite limited.

2023/10/10

Risk–based Management of Energy Infrastructure132(1) Uncertainty of flaw detection
POD (Probability of Detection)

No need to detect all flaws.
min. detectable flaw size

detectability

100%0%

2023/10/10

real

harmless ideal
flaw

harmfulflaw (=defect)

flaw size
Risk–based Management of Energy Infrastructure18

2(1) Uncertainty of flaw detection
early POD studies (70’s)
Directly, the probability of detection isgiven asdetected

number of flaws

The lower confidence interval of thisprobability can be calculated given as

BGW. Yee et al., Assessment of NDE Reliability Data,
NASA-CR-134991 (1976)
2023/10/10

Risk–based Management of Energy Infrastructure192(1) Uncertainty of flaw detection
early POD studies (70’s)
Directly, the probability of detection isgiven as
detected

number of flaws

The lower confidence interval of thisprobability can be calculated given as

BGW. Yee et al., Assessment of NDE Reliability Data,
NASA-CR-134991 (1976)
2023/10/10

Risk–based Management of Energy Infrastructure23

2(1) Uncertainty of flaw detectionMore recent POD studies
Results obtained by the approach based on the binomial
distribution, which used in the early study, significantly depend on
how to categorize flaws.

Estimating the parameters of afunction that represents a probabilityof detection
 Binary data (detected or undetected)→Hit/Miss approach Signal amplitude > Threshold?
→â-a approach

2023/10/10

Risk–based Management of Energy Infrastructure242(1) Uncertainty of flaw detection
Recent POD: Hit/Miss approach[For binary data]
Detected

Undetected(missed)2023/10/10

Flaw size
Risk–based Management of Energy Infrastructure26

2(1) Uncertainty of flaw detection
Recent POD: Hit/Miss approach[The early approach]
Detected0%

Undetected(missed)2023/10/10

50%

33%

33%

100%

Flaw size
Risk–based Management of Energy Infrastructure27

2(1) Uncertainty of flaw detection
Recent POD: Hit/Miss approach[Estimating the parameters of a function]

Probability

Detected

Undetected(missed)2023/10/10

Flaw size
Risk–based Management of Energy Infrastructure28

2(1) Uncertainty of flaw detection
Recent POD: Hit/Miss approach[Problem]

Probability

Detected

Undetected(missed)

2023/10/10

Flaw size
Risk–based Management of Energy Infrastructure29

2(1) Uncertainty of flaw detection
Recent POD: Hit/Miss approach[Problem]

Probability

Detected

Undetected(missed)

2023/10/10

Flaw size
Risk–based Management of Energy Infrastructure30

2(1) Uncertainty of flaw detection
Recent POD: â-a approach

Flaw size, a①Measurement

Flaw size, a②Regression

100P(â≧âth)POD [%]

â=β0+β1a+N(0,s2)

Signal response, âSignal response, âSignal response, â[For “signal amplitude” data]âthFlaw size, a③Threshold

0Flaw size, a④POD• Applicable to the results where the results ofmeasurements are given as numeric data (not binary).
• Capable of considering the effect of the decisionthreshold2023/10/10

Risk–based Management of Energy Infrastructure31

2(1) Uncertainty of flaw detection
Recent POD: â-a approach

Flaw size, a①Measurement

Flaw size, a②Regression

100P(â≧âth)POD [%]

â=β0+β1a+N(0,s2)

Signal response, âSignal response, âSignal response, â[For “signal amplitude” data]âthFlaw size, a③Threshold

0Flaw size, a④POD• Applicable to the results where the results ofmeasurements are given as numeric data (not binary).
• Capable of considering the effect of the decision𝑎 − 𝛽 +𝛽 𝑎1𝑎 − 𝛽 +𝛽 𝑎
𝛽 +𝛽 𝑎 −𝑎𝐿=Φ×exp×Φ
threshold
𝜎𝜎𝜎2𝜋𝜎2023/10/10

Risk–based Management of Energy Infrastructure35

2(1) Uncertainty of flaw detection
Recent POD: â-a approach

Flaw size, a①Measurement

Problems••••2023/10/10

Flaw size, a②Regression

100P(â≧âth)POD [%]

â=β0+β1a+N(0,s2)

Signal response, âSignal response, âSignal response, â[For “signal amplitude” data]âthFlaw size, a③Threshold0

Flaw size, a④POD

Simple linear regression model
Constant variance is necessary
Many experimental signals (&samples) needed
A single parameter to characterize a flaw
Risk–based Management of Energy Infrastructure36

2(1) Uncertainty of flaw detection
Our approach for reconstructing the â-a approach Simple regression model→Not closed-form Constant variance→Variance depending on flaw size Many experimental signals (&samples)→Combinational use of measurements andsimulations
 A single parameter to characterize flaw
→Multiple flaw parameters2023/10/10

Risk–based Management of Energy Infrastructure37

2(1) Uncertainty of flaw detection① detecting surface cracks by eddy current testing
1. Regression analysis based on the combinational use of measured and simulated signals
Normal distribution

MeasuredDepth

Simulated

Normal distribution

Length

2. Maximum likelihood analysis for estimating the parameters
𝑉 − 𝜇 𝑉ln 𝐿 =ln Φ+ln 1 − Φ𝑉(𝑑 , 𝑙 ) + 𝜇(𝑑 , 𝑙 ) 𝜎 + 𝜎
𝑉 − 𝜇 𝑉𝑉−12ln 2𝜋 𝑉(𝑑 , 𝑙 ) 𝜎 + 𝜎+𝑉 − (𝜇 𝑉 (𝑑 , 𝑙 ) + 𝜇 )𝑉 (𝑑 , 𝑙 ) 𝜎 + 𝜎(𝑑 , 𝑙 ) + 𝜇(𝑑 , 𝑙 ) 𝜎 + 𝜎

3. Probability of detection given as the probability that measured signal exceeds a threshold• N. Yusa et al, Demonstration of probability of detection taking consideration of both the length and thedepth of a flaw explicitly, NDT&E International 81 (2016), 1-8.
2023/10/10

Risk–based Management of Energy Infrastructure38

2(1) Uncertainty of flaw detection① detecting surface cracks by eddy current testingTwo-dimensional POD

90% POD + Confidence interval• N. Yusa et al, Demonstration of probability of detection taking consideration of both the length and thedepth of a flaw explicitly, NDT&E International 81 (2016), 1-8.
2023/10/10

Risk–based Management of Energy Infrastructure39

2(1) Uncertainty of flaw detection② effect of the distance between scanning lines 36 fatigue cracks on type 316L SS pluspoint probe, 100kHz
 one flaw parameter model scanning line runs parallel to a crack

Length & depth of the fatigue cracks

Depth vs FWHM of signal distribution(when probe runs perpendicular to a fatigue crack)• N. Yusa et al., Probabilistic evaluation the area of coverage of a probe used for eddy current non-destructive
inspections, International Journal of Applied Electromagnetics and Mechanics 64 (2020), 11-18.
2023/10/10

Risk–based Management of Energy Infrastructure40

2(1) Uncertainty of flaw detection② effect of the distance between scanning lines
Then, evaluatex⁄⁄⁄s⁄distance)

signalby the Monte-Carlo method to evaluate signal.

s=3 mms=5 mms=7 mmProbability of detection with variable distances• N. Yusa et al., Probabilistic evaluation the area of coverage of a probe used for eddy current non-destructive
inspections, International Journal of Applied Electromagnetics and Mechanics 64 (2020), 11-18.
2023/10/10

Risk–based Management of Energy Infrastructure41

2(1) Uncertainty of flaw detection③ effect of sensor placing in wall thinning monitoring
The POD is given as the probability[Assumption1]

Fig. The dimensions and sensor placements[Assumption2]& The four parameters were estimated by comparing experimental andnumerical signals due to 27 (i = 1, 2, ..., 27) samples.
 The effect of xA and xC were evaluated by Monte-Carlo simulations(N=1,000,000).
 The confidence interval of POD was calculated by the bootstrap
method (1,000 samples).
2023/10/10• H. Song, N. Yusa, A probability of detection model for a sensor-based monitoring method against local wall
thinning, International Journal of Applied Electromagnetics and Mechanics 71 (2023), S29-S37.
Risk–based Management of Energy Infrastructure422(1) Uncertainty of flaw detection③ effect of sensor placing in wall thinning monitoring

Fig. POD contour when SA=50 mm and SC=90°

Fig. POD contour when SA=50 mm and SC=60°

2023/10/10• H. Song, N. Yusa, A probability of detection model for a sensor-based monitoring method against local wall
thinning, International Journal of Applied Electromagnetics and Mechanics 71 (2023), S29-S37.
Risk–based Management of Energy Infrastructure43

2(1) Uncertainty of flaw detection④ from POD to ROC
A lower ath leads not only higher POD but also a higher PFA

Noise distribution

Signal distribution

Threshold→Necessity to consider both POD and PFA• F. Yu et al, Receiver operating characteristic analysis for evaluating a proper experimental condition of eddy current tests
under a low signal-to-noise ratio, International Journal of Applied Electromagnetics and Mechanics 71 (2023), S179-S189.
2023/10/10

Risk–based Management of Energy Infrastructure44

2(1) Uncertainty of flaw detection④ from POD to ROC• F. Yu et al, Receiver operating characteristic analysis for evaluating a proper experimental condition of eddy current tests
under a low signal-to-noise ratio, International Journal of Applied Electromagnetics and Mechanics 71 (2023), S179-S189.
2023/10/10

Risk–based Management of Energy Infrastructure45

2(2) Uncertainty of flaw evaluation
Evaluating (sizing) a flaw is an inverse problem: forward problem: inverse problem
ill-posedness of inverse problems (from NDT viewpoint) there would be no that gives , (necessity ofproper flaw modeling);
 two provides the same ; small change in would lead to a large changein (small noise would lead to a large error).
2023/10/10

Risk–based Management of Energy Infrastructure47

2(2) Uncertainty of flaw evaluation
Evaluating (sizing) a flaw is an inverse problem: forward problem: inverse problem
ill-posedness of inverse problems (from NDT viewpoint) there would be no that gives , (necessity ofproper flaw modeling);
 two provides the same ; small change in would lead to a large changein (small noise would lead to a large error).
2023/10/10

Risk–based Management of Energy Infrastructure48

2(2) Uncertainty of flaw evaluation
An earlier study• N. Yusa et al., Caution when applying eddy current inversion to stress corrosion cracking, Nuclear
Engineering and Design 236 (2006), 211-221.
2023/10/10

Risk–based Management of Energy Infrastructure49

2(2) Uncertainty of flaw evaluation
An earlier study ~ huge error!!

The estimated profile differed significantly from the true one,
although the signal was well reproduced.

→ Necessity to evaluate the uncertainty• N. Yusa et al., Caution when applying eddy current inversion to stress corrosion cracking, Nuclear
Engineering and Design 236 (2006), 211-221.
2023/10/10

Risk–based Management of Energy Infrastructure50

2(2) Uncertainty of flaw evaluation
Evaluating (sizing) a flaw is an inverse problem: forward problem: inverse problem
ill-posedness of inverse problems (from NDT viewpoint) there would be no that gives , (necessity ofproper flaw modeling);
 two provides the same ; small change in would lead to a large changein (small noise would lead to a large error).
2023/10/10

Risk–based Management of Energy Infrastructure51

2(2) Uncertainty of flaw evaluation
Evaluating possible error range ~ an earlier approachCalculating all possible (flaw profile)-(signal) combinations.

Model (flaw on a t10 plate）(a) Flaw model1(b) Flaw model2

Possible error caused by 10% signal difference Small error would lead to a large error in flaw evaluation→Point estimation is insufficient• N. Yusa et al, Numerical evaluation of the ill-posedness of eddy current problems to size real cracks, NDT&E International 40(2007), 185-191.
• N. Yusa, H. Hashizume, Numerical investigation of the ability of eddy current testing to size surface breaking cracks,
Nondestructive Testing and Evaluation 32 (2017), 50-58.
2023/10/10

Risk–based Management of Energy Infrastructure52

2(2) Uncertainty of flaw evaluation
More quantitative approach
The Bayes’ theorem states
posterior distribution

flaw profile

likelihood

signalprior distribution

from POD analysisThus,• C. Cal et al., Metamodel-based Markov-Chain-Monte-Carlo parameter inversion applied in eddy
current flaw characterization, NDT&E International 99 (2018), 13-22.
• T. Tomizawa and N. Yusa, Bayesian data fusion of eddy current testing for flaw characterization withuncertainty evaluation, NDT&E International (under review)2023/10/10

Risk–based Management of Energy Infrastructure54

3. Summary
PRA/PFM
RBI
Prediction/
Planning

Repair/Replacement

POD
Profile Likelihood
Nondestructive
Inspection
Inspection

Evaluation

????

PFM?

Fig. General flow of maintenance activity
2023/10/10

Risk–based Management of Energy Infrastructure60

Thank you for your attention.

2023/10/10

Risk–based Management of Energy Infrastructure61