E-Journal of Advanced Maintenance Vol.11 No.2 (2019) pp.65-78
Japan Society of Maintenology

Evaluation of Capabilities on Ultrasonic Testing Examiners Using
Probability of Defect Detection and Cumulative Failure Probability
Masayoshi KOJIMA1,*, Hideharu TAKAHASHI1 and Hiroshige KIKURA1
1

Tokyo Institute of Technology, 2-12-1-N1-7 Ookayama, Meguro-ku, Tokyo 152-8550, Japan

ABSTRACTUltrasonic testing (UT) is one of the non-destructive testing for pressure-retaining weld lines in class 1 piping of
nuclear power plants. Since capabilities of UT examiners may affect failure risks of class 1 piping, UT is
performed by certified examiners. In this paper, capabilities of uncertified and certified examiners were
evaluated using probability of detection (POD) models and cumulative failure probabilities. The POD models is
one of the input data of probabilistic fracture mechanics (PFM) analysis. Experimental data of blind testing fordetecting stress corrosion cracking conducted by the uncertified and certified examiners were used for
regression analysis of POD model. Two types of equation were selected for the POD model. Cumulative failure
probabilities considering several inspection intervals were calculated in the PFM analysis. Then, the authors
evaluated the failure risks of piping performed by the uncertified and certified examiners using the POD models
and the cumulative failure probabilities. As a result, one equation of POD model with low residual sum ofsquares made it easier to evaluate the capabilities of UT examiners than the other POD model. Additionally, theequation with low residual sum of squares had less uncertainty on the failure risks of piping. Furthermore, it was
shown that UT results performed by certified examiners might have equivalent failure risks of piping as UT
results performed by uncertified examiners if an inspection interval gets longer.
KEYWORDS

ARTICLE INFORMATION

Failure Risk of Piping, Probability of Detection, Probabilistic Fracture
Mechanics, Cumulative Failure Probability, Capabilities of Examiners,
Ultrasonic Testing, Stress Corrosion Cracking, 95% Confidence Intervals

Article history:
Received 7th May 2019
Accepted 30th August 20191.

Introduction

Piping in nuclear power plants (NPPs) is generally subjected to non-destructive testing for theirmaintenance management. Ultrasonic testing (UT) is conducted at pressure-retaining weld lines ofclass 1 piping in NPPs [1]. Since capabilities of UT examiners may affect failure risks of class 1
piping, the UT is performed by certified examiners. Probabilistic fracture mechanics (PFM) analysis
is one of the useful methods to evaluate the failure risks of piping. The PFM analysis has been studied
for considering an inspection interval of the UT in a piping of primary loop recirculation (PLR)
system of a boiling water reactor (BWR) in recent years. Machida [2] showed effects of probability
factors, which were related to the JSME S NA1-2008 (Rules on Fitness-for-Service for Nuclear Power
Plants) [3], on a reliability of the PLR piping. Cumulative failure probabilities were calculated using
several virtual models of probability of detection (POD) in the PFM analysis. The conclusion showedreliable detection to large defect size is important to avoid piping failure. Shohji et al. [4] showed
effects of cumulative failure probabilities in a case where the maximum POD was less than 1.0(100%). Cumulative failure probabilities were also calculated using several virtual POD models. The
conclusion showed the cumulative failure probabilities of the PLR piping were low if the defectheight was around 8 mm or less even though the maximum POD was 0.9 (90%). However, the authors
considered that the necessity of an evaluation regarding the effect on the failure risks of piping due tothe difference in the capabilities of the UT examiners using POD model of experimental data has beenremained. The POD is generally modeled by the regression analysis using results of defect detection
measured by the UT examiners [5]. Several previous studies referred the POD models using UT
results of blind testing for weld lines of austenitic stainless steel piping [2, 5, 6]. The results of blind*

Corresponding author, E-mail: kojima.m.al@m.titech.ac.jp

ISSN-1883-9894/10 © 2010 – JSM and the authors. All rights reserved.

65

M. KOJIMA, et al./ Evaluation of Capabilities on Ultrasonic Testing Examiners
Using Probability of Defect Detection and Cumulative Failure Probability

testing are related to UT examiners’ capabilities; therefore, it is necessary to grasp effects on the
failure risks of piping using experimental data of blind testing considering the UT examiners’
capabilities.
The objective of this paper is to evaluate effects on the failure risks of piping using experimental
data considering the difference of the UT examiners’ capabilities. First, the authors organizedexperiments of UT examinations to obtain experimental data of uncertified examiners. The uncertified
examiners conducted the experiments for detecting stress corrosion cracking (SCC) at weld lines ofpiping. An experimental data of previous study for detecting SCCs conducted by certified examiners[7] were investigated and utilized as well. Then, the authors calculated POD models using bothexperimental data. Two types of equation were selected for the POD models. In addition, cumulative
failure probabilities were calculated in the PFM analysis considering several inspection intervals.
Finally, the authors evaluated effects on the failure risks of piping using the cumulative failure
probabilities and the POD models considering the difference of the UT examiners' capabilities.

2.

Methods

2.1. Experiments of Ultrasonic testing examinations
The UT examinations of the uncertified examiners were performed by 9 university students withno experience of manual UT for detecting SCC. The examiners conducted blind testing in which thelocations and size of defects were not informed them. All the uncertified examiners participated in an
UT training program organized by Nondestructive Evaluation Center of the Japan Power Engineering
Inspection Corporation (JPEIC). The training was scheduled before the blind testing. Table 1 shows
the outline of the training program. The training for detecting fatigue cracking in the carbon steel
piping and SCCs in austenitic stainless steel piping was organized for 24 hours in total.
Table 1 Outline of lecture and practical training program organized by Nondestructive Evaluation Centerof the JPEIC for uncertified examiners before conducting the experiments
Items

Hours

Contents

Lecture Basics of ultrasonic testing (UT)

1. Basics of UT
2. UT instrument

Lecture Outline of UT

1. Procedure of UT
2. Calibration of UT instrument

3.0
3. Angle beam technique
4. Analysis of results

3.0

Practice

Calibration of UT instrument in angle 1. Probe index and angle of refraction by STB 3. Calibration sensitivity and distance-amplitude
beam technique
2. Calibration of measurement range
correction

1.5

Practice

UT to welded part of carbon steel
plate

4.5

Lecture Outline of UT to weld of austenitic
Practice stainless steel (SUS) piping
Practice

1. Angle beam technique
2. Data collection

3. Analysis

1. Characteristic of UT on weld of SUS
2. Stress Corrosion cracking (SCC)

3. Important points on UT of SUS
4. Difference between carbon steel and SUS

2.0

3. Calibration sensitivity and distance-amplitude
correction

1.0

3. Analysis
4. Extraction of defect echo

6.0

Calibration of UT instrument in angle 1. Conversion of refraction angle
beam technique
2. Re-calibration of measurement range

Practice UT to weld of SUS piping

1. Angle beam technique
2. Data collection

LectureFeedback on the results of practice
Practice

1. Feedback on the results of practice
2. Explanation of key points in SCC detection

LectureSecondary creeping wave
Practice

1. Principle, application and role
2. Calibration of UT instrument

1.5
3. Important points on handling of probe

1.5
Total

24.0

Table 2(a) shows the examination conditions of the blind testing conducted by the uncertified
examiners [8, 9]. The room temperature of an air conditioner during the blind testing was set to 24℃.
SCCs at the welding heat affected zone (HAZ) in 350A piping of the SUS304 grade stainless steel
were detected in the blind testing. 10 test specimens were prepared for the blind testing. 11 locations

66

M. KOJIMA, et al./ Evaluation of Capabilities on Ultrasonic Testing Examiners
Using Probability of Defect Detection and Cumulative Failure Probabilityof SCC in 7 test specimens were existed. There was no SCC in 3 test specimens. The 11 locations of
SCC were identified as UE001 to UE011. Each examiner randomly selected 4 test specimens in theblind testing.
Table 2 Examination conditions of UT by (a) uncertified examiners and (b) certified examiners forcalculations of POD
Items(a) Uncertified Examiners(b) Certified Examiners [7]Examiner

9 university students

6 teams

Inspection guideline

JEAC 4207-2008 [8]JEAC 4207-2004 [10]Manual scanning

Manual scanning: 5 teams
Automatic scanning: 1 teamSingle probe techniqueAngle beam techniqueSingle probe techniqueAngle beam technique and normal beam technique

Digital UT instrument(UI-25IIb (Ryosho Electronics))

Digital or analog UT instrument

Probe

Shear wave normal probe
Frequency:2MHz
Mode: Shear wave
Angle: 45deg
Element size: 10x10mm(Type: 2Z10x10A45 (Japan Probe) or 2C10x10A45 (KGK))

Shear wave normal probe
Frequency:2MHz
Mode: Shear wave
Angle: 45deg
Element size: 10x10mm(Type: 2Z10x10A45)

Couplant

Glycerin paste
Glycerin paste(SONI COAT BS-400 (Taiyo Nippon Sanso Gas & Welding))

Scanning
Probe techniqueInstrument

Calibration block

STB-A1 (JIS Z 2345-1 [9])STB-A1 (JIS Z 2345-1 [9])Reference block

SUS304, 25t (Side drilled hole: φ2.4x1)

P350A-25t-SUS304 (Side drilled hole: φ2.4x1)

Stainless steel piping welds specimen (350A, t25)
Material: SUS304 grade stainless steel
Size: φ355.6mm x L400mm x 14.5deg(≒W90mm)

Stainless steel piping welds specimen (350A, t25)
Material: SUS304 grade stainless steel
Size: φ355.6mm x L1200mm

Test specimen

Welding procedure: TF + A
Defect: SCC (Position: Heat-affected zone)
Number of specimens: 10
Number of SCCs: 11 (ID: UE001 ~ UE011)
Weld groove shape:

Welding procedure: Defect: SCC (Position: Heat-affected zone)
Number of specimens: Number of SCCs: 12 (ID: PSS21 ~ PSS29, PSS2A ~ PSS2C)
Weld groove shape:

Testing period

About 1 hour / test specimen / person

1 week / teamThe largest defect height measured by 3 types of probes was defined as the defect height of the
11 locations of SCC; where 3 types of probes were the shear wave end echo method using a fixed
angle probe, the longitudinal wave end echo method using a fixed angle probe, and the longitudinal
wave end echo method using a phased array probe. Although 2 locations of SCC (UE006 and UE009)
were not detected by either of the 3 type of probes, they were detected by the liquid penetrant testing.
The authors defined the defect height of the 2 SCCs as 1.5 mm; since the wavelength of 1 cycle in the
UT examination is calculated to be 1.55 mm from values of the shear wave velocity in the austenitic
stainless steel (3,100 m/s) and the frequency of the shear wave probe (2MHz), the authors predicted
the wavelength of 1 cycle was the detection limit in the examination conditions. The authors expected
the defined defect height of the 2 SCCs was 0.05 mm smaller than the predicted detection limit.
Procedure for detecting SCCs followed the JEAC 4207-2008; therefore, the uncertified
examiners adjusted a sensitivity of the ultrasonic instrument, searched a unit exit point and a beam
angle of a probe, drew a distance amplitude correction (DAC) curve, recorded all point of levels

67M. KOJIMA, et al./ Evaluation of Capabilities on Ultrasonic Testing Examiners
Using Probability of Defect Detection and Cumulative Failure Probability

greater than 20% of a DAC curve, and evaluated all the suspected flaw indications considering thepoint of reflection for detecting a SCC.
Experimental data for detecting SCCs conducted by certified examiners, which were organizedby the Japan Nuclear Energy Safety Organization (JNES) [7], were investigated and utilized forcomparison with the results of the uncertified examiners. Table 2(b) shows the examination conditionsof UT by 6 teams of the certified examiners [7, 9, 10]. All the certified examiners had the certificationof the non-destructive testing of UT Level 2 or 3, and have had experiences of in-service inspection(ISI) in the NPPs [7]. Test specimens with 12 locations of SCCs at the HAZ were examined [7].
2.2. Models of probability of detection
The POD model was defined as an equation of a defect height a. 2 types of equation of POD
models with 1 regression coefficient and 2 regression coefficients were used for input data of the PFM
analysis. The equation with 1 regression coefficient was defined as Model-1 described by Eq. (1) [11].
The equation with 2 regression coefficients was defined as Model-2 described by Eq. (2) [5].
Model-1

POD (a )  1  exp a   
 0 ,  95% CL ,  95% CLse ,Model-2(1)

Se

POD(a)  1  exp  1   2  a 
 10 ,  195%CL ,  1 95%CL

1

 20 ,

 295%CL ,se 1 ,se 2 ,

(2)

 2 95%CL

Se

where, a (mm) is a defect height of SCC. β, β1, and β2 are regression coefficient. β0, β10, and β20 are
regression coefficient that minimized residual sum of squares. β-95%CL, β+95%CL, β1-95%CL, β1+95%CL,
β2-95%CL, and β2+95%CL are regression coefficient in 95% confidence intervals. seβ, seβ1, and seβ2 arestandard errors. Se is a residual sum of squares in β0, β10, and β20.
1.00

Se + F×

β-95％CL = β0 - δ3

β+95％CL = β0 + δ4

0.90
0.80

seβ = ( δ1 + δ2 ) / 2
Se 0.70
+ Ve
Se

0.60
0.50

‐0.6

-δ3

‐0.4

-δ1
‐0.2

00

δ20.2

δ4 0.4

0.6

Change of the regression coefficient δ

Fig. 1. Concept of profile likelihood approach for calculating the standard error and 95% confidence
intervals

Fig. 1 shows a concept of profile likelihood approach for calculating the standard error and 95%
confidence intervals [12]. Se (the residual sum of squares) is the sum of the squared difference
between all the experimental data and equation of POD model calculated by the regression analysis.

68

M. KOJIMA, et al./ Evaluation of Capabilities on Ultrasonic Testing Examiners
Using Probability of Defect Detection and Cumulative Failure Probability

Se (the standard error) is calculated from -δ1 and δ2 which are the negative / positive change of theregression coefficients at Se + Ve (Ve = Se / n: mean square for error, n: degrees of freedom of residualin the equation). The standard error is the arithmetic mean of δ1 and δ2; therefore, the calculated
standard error called the approximate standard error [12]. The 95% confidence intervals are calculated
from -δ1 and δ2 at Se + F × Ve (F = F0.05(1, n): Snedecor's F-distribution at 0.05 significance level with
1 numerator degrees of freedom and n denominator degrees of freedom). The 95% lower confidence
intervals are the sum of the regression coefficient and -δ3, and the 95% upper confidence intervals are
the sum of the regression coefficient and δ4.
The algorithms for the calculation of the standard error and 95% confidence intervals using the
profile likelihood approach followed the Appendix I and II of the "Application of Nonlinear
Regression Model to Sigmoid Dose-Response Relationship in Pharmacological Studies" [13]; here,
the equation of sigmoid function using 4 regression coefficients (β1, β2, β3, and β4) in those Appendices
was replaced by Eq. (1) and Eq. (2) in the calculation process. The generalized reduced gradient(GRG) nonlinear method built in Microsoft Office Excel solver add-in was used in those calculations.
2.3. Calculation of cumulative failure probability
Cumulative failure probabilities of a SCC growth in a piping were calculated using a PFM
analysis software. The software was written by the authors in visual basic for applications (VBA) built
in Microsoft Office Excel. The software was referred the JSME S NA1-2008 [3] and previous studies[2, 4, 6, 14-20], and was verified comparing the analysis results of previous studies [6, 16].
Table 3 Analysis conditions of PFM for calculation of cumulative failure probabilities
Items
Analyzed pipe
(referring to Fig. 2)
Failure mode
Stress intensity factor
K (MPa√m)
Crack height reaching weld metaldc (mm) (referring to Fig. 2)
Distance from weld metal to a SCC
L (mm) (referring to Fig. 2)
Adding lengthα (mm)

Analysis Conditions
400A Sch100 (16B Sch100)
Pipe outside diameter: D O = 406.4 (mm)Wall thickness: t = 26.2 (mm)Material: 316Lgrade stainless steel
Stress Corrosion Cracking (SCC)
JSME S NA1-2008 (Att.E-5 5.3(1))

SCC growth rate
in weld metal
da /dt (m/s)(referring to Fig. 2)

Normal operating loads (MPa)[3]dc = L +α[2, 16, 17]Median: μL = 1.15
Standard deviation: σL = 1.39 (Log-normal distribution)[2, 16, 17]Median: μα = 2.99
Standard deviation: σα = 1.31 (Normal distribution)[2, 16, 17]m

SCC growth ratein heat-affected zone (HAZ)da /dt (m/s)(referring to Fig. 2)

References

da/dt = C ･K
Exponent: m = 2.161 (Constant)Coefficient: C
C = 0 (K<0)-14Median: μCHAZ = 9.018x10(K≧0)

Standard deviation: σCHAZ = 0.303 (Log-normal distribution)mda/dt = C ･K
Exponent: m = 2.161 (Constant)Coefficient: C
C = 0 (K<0)-14Median: μCWM = 1.017x10(K≧0)

Standard deviation: σCWM = 1.120 (Log-normal distribution)
Internal pressure p = 9.0
Membrane stress σm = 34.9
Bending stress σb = 10.0
Thermal expansion stress σe = 40.0

69

[2, 3, 16][2, 3, 16][3][2, 16][2, 3, 16][2, 3, 16][3][2, 16][16]

M. KOJIMA, et al./ Evaluation of Capabilities on Ultrasonic Testing Examiners
Using Probability of Defect Detection and Cumulative Failure Probability

Table 3 shows the analysis conditions of PFM for calculating cumulative failure probabilities [2,3, 16, 17]. Eq. (1) and Eq. (2) of both uncertified and certified examiners were used for the analysis
input conditions as well. The piping material was defined SUS316L grade stainless steel [6, 16].
Fig. 2 shows a concept of a SCC growth in a HAZ and a weld metal of the SUS316L grade
stainless steel piping [2, 16, 17]. The thickness (26.2 mm) of the 400A piping in the PLR system of
BWR was close to the thickness (25.0 mm) of the 350A piping in the UT blind testing. Since thefatigue crack growth is small, that was not considered in the PFM analysis [4].
Stress due to welding residual stress, internal pressure, membrane stress, bending stress andthermal expansion stress were considered in the main factors contributing to the SCC growth [2, 3,16]. Welding residual stress distribution in 400A austenitic stainless steel piping of the PLR system in
BWR was utilized from the results of finite element method (see Fig. 3) [18, 19].
The authors defined an initial defect size as 0.01 mm in depth and 0.02 mm in length in the PFM
analysis. Extrapolated values at the aspect ratios of 0.1 and 0.2 were used for the correction factor G[3] of the surface defect if the aspect ratio was less than 0.1 [19]. The failure of piping (leak or breakof the piping) was defined in a case where the defect height was larger than the 75% of wall thickness[2, 4, 6].
The operating period was defined 40 years [6, 16]. The inspection intervals of 4, 5, 8, and 10
years were selected for the analysis input conditions. Pre-service inspection was not considered for the
analysis. Time step size was 0.1 years. Sample size of Monte Carlo simulation was 106. The Failure
probability per 1 sample was defined the calculation result of multiplying all of the probability of
non-detection (PND: 1- POD) at the time of inspection in a case where the piping was failure [14, 17,20]. The cumulative failure probabilities were calculated in the PFM analysis.

Fig. 2. Concept of SCC growth in HAZ and weld metal of SUS316L grade stainless steel piping [2, 16, 17]

Welding Residual Stress (MPa)

500
400
300
200
100
0
-100
-200
0

5

10

15

20

25

30

Distance from Inner Surface (mm)Fig. 3. Welding residual stress distribution of 400A austenitic stainless steel piping in PLR system [18, 19]70

M. KOJIMA, et al./ Evaluation of Capabilities on Ultrasonic Testing Examiners
Using Probability of Defect Detection and Cumulative Failure Probability

3.

Results

3.1 Ultrasonic testing examinations
Table 4 shows the examination results of UT conducted by (a) the uncertified examiners and (b)the certified examiners [7].
Table 4 Examination results of UT for SCCs in SUS304 grade stainless steel 350A piping conducted by (a)uncertified examiners and (b) certified examiners(a) Uncertified examiners(b) Certified examiners [7]ID

Defect heighta (mm)

Probability of defect detection
POD (a )ID

Defect heighta (mm)

Probability of defect detection
POD (a )

UE001

7.5

1.00

PSS21

2.0

1.00

UE002

2.0

1.00

PSS22

2.1

1.00

UE003

8.0

1.00

PSS23

3.9

1.00

UE004

2.0

0.60

PSS24

2.4

1.00

UE005

1.8

0.50

PSS25

2.0

1.00

UE006

1.5

0.00

PSS26

2.4

1.00

UE007

4.2

1.00

PSS27

0.6

0.17

UE008

5.8

0.75

PSS28

3.6

1.00

UE009

1.5

0.00

PSS29

3.3

1.00

UE010

2.0

1.00

PSS2A

1.4

1.00

UE011

4.3

0.80

PSS2B

2.7

1.00

PSS2C

0.7

0.00

3.2 Probability of detection
Eqs. (3)-(6) show the equation of the Model-1 and Model-2 calculated from the examination
results of UT conducted by the uncertified and certified examiners in Table 4.
Model-1 of the uncertified examiners

POD(a )  1  exp a   

 0  1.17,

 95%CL  0.649,

se  0.184,

Se  0.625

 95%CL  1.51

(3)

Model-2 of the uncertified examiners

POD(a)  1  exp  1   2  a 
 10  19.0,  1 95%CL  N/A,

1

 20  10.5,

 295%CL  4.93,

se 1  8.51,

se 2  4.67,

71

 1 95%CL  8.72
 2 95%CL  N/A

Se  0.213(4)

M. KOJIMA, et al./ Evaluation of Capabilities on Ultrasonic Testing Examiners
Using Probability of Defect Detection and Cumulative Failure Probability

Model-1 of the certified examiners

POD(a )  1  exp a   
 0  0.331,  95%CL  0.0938,
se  0.159,

 95%CL  0.630(5) 1 95% CL  5.73
 2 95%CL  N/A(6)

Se  0.363

Model-2 of the certified examiners

POD(a)  1  exp  1   2  a 
 10  8.82,  1 95%CL  N/A,

1

 20  8.93,

 2 95%CL  5.75,

se 1  3.71,

se 2  4.41,

Se  0.0252

where, N/A stands for the not available; since the change of the regression coefficient δ did not reachSe + F × Ve, the 95% confidence intervals were not calculated (see Figs. 4 and 5).
0.4
0.4

0.4
0.4

0.38

0.38
0.36

0.36

Se + F× Ve

0.34

0.334

0.34

0.334

0.32

0.32

0.3

0.3

0.28

0.28
0.26

0.26

Se + Ve

0.24
0.237

‐20

‐15-11.3
‐10

‐5

5
10
10.2
5.75

00

15

20

0.24

Se + Ve

0.22

Se

0.237

Se

0.22

0.213
0.2
0.2
‐25
-25

Se + F× Ve

0.213
0.2
0.2
‐15
-15

25
25

‐12

‐9

‐6
‐3
-5.54
-3.07

00

3

6
6.26

9

Change of the regression coefficients δChange of the regression coefficients δ(a) β1(b) β212

15
15

Fig. 4. Profile likelihood approach for calculating 95% confidence intervals of (a) β1 and (b) β2 in Eq. (4)
0.06
0.06

0.06
0.06

0.055

0.055

0.05

0.05

0.045

0.045

0.04

0.04

Se + F× Ve

0.0376

0.0376

0.035
0.03

0.03

Se + VeSe

0.02770.025
0.0252
0.02
0.02
‐10
-10

Se + F× Ve

0.035

‐8

‐6
-5.65

‐4

‐2

0

2 3.09
4
1.77

6

8

0.02770.0252
0.025
0.02
0.02
‐10
-10

10
10

Change of the regression coefficients δ

Se + Ve
Se

‐8

‐6

‐4
‐2
-1.90
-3.18

00

2

4

6 6.92 810
10Change of the regression coefficients δ(a) β1(b) β2

Fig. 5. Profile likelihood approach for calculating 95% confidence intervals of (a) β1 and (b) β2 in Eq. (6)72M. KOJIMA, et al./ Evaluation of Capabilities on Ultrasonic Testing Examiners
Using Probability of Defect Detection and Cumulative Failure Probability

Although β1-95%CL and β2+95%CL were not calculated in Eqs. (4) and (6), equations of the 95%
confidence intervals are described as Eqs. (7) and (8). This is because the residual sum of squares in
Eqs. (7) and (8) are as equal as the Se + F × Ve in Eqs. (4) and (6).
Model-2 of the uncertified examiners at 95% confidence intervals in Eq. (4)POD(a )  1  exp  1 95%CL   295%CL  a 

1 1 95%CL  8.72,

 2 95%CL  4.93

(7)Se95%CL  Se  F  Ve  0.213  5.12  0.0237  0.334
n  nUE  n1  11  2  9Model-2 of the certified examiners at 95% confidence intervals in Eq. (6)POD(a)  1  exp  1 95% CL   2 95%CL  a 

1 1 95%CL  5.73,

 2 95%CL  5.75

(8)Se95%CL  Se  F  Ve  0.0252  4.96  0.00252  0.0376
n  nCE  n1  12  2  10

where, Se95%CL is the residual sum of squares in Eqs. (7) and (8). Se is the residual sum of squares in
Eqs. (4) and (6) in β10 and β20. Ve is the mean square for error in Eqs. (4) and (6) in β10 and β20. F is
the Snedecor's F-distribution in Eqs. (4) and (6) in β10 and β20. nUE is the number of SCC locations in
UT examination by uncertified examiners. nCE is the number of SCC locations in UT examination bycertified examiners. n1 is the number of the regression coefficient in Eqs. (4) and (6).

1.2

1.2

1.0

1.0

0.8

Probability of Defect Detection (―)

Probability of Defect Detection (―)

Fig. 6 (a) shows the PODs of the Model-1 described by Eqs. (3) and (5). Fig. 6 (b) shows the
PODs of the Model-2 described by Eqs. (4) and (8). The horizontal axis is the defect height a (mm).
The vertical axis is the POD. UE stands for the uncertified examiners, and CE stands for the certified
examiners. The red circles are the examination results conducted by the uncertified examiners. The
yellow triangles are the examination results conducted by the certified examiners.

UE011
UE008

UE: Uncetified Examiners
CE: Certified Examiners
95%CI: 95% Confidence Iintervals

0.6

0.4

UE Eq. (3)UE Eq. (3) ±95%CI
CE Eq. (5)CE Eq. (5) ±95%CI
Uncertified Examiners
Certified Examiners [7]

0.2

0.0

0.8

UE011
UE008

UE: Uncetified Examiners
CE: Certified Examiners
95%CI: 95% Confidence Iintervals

0.6

0.4

UE Eq. (4)UE Eq. (7) (: Eq. (4) 95%CI)
CE Eq. (6)CE Eq. (8) (: Eq. (6) 95%CI)
Uncertified Examiners
Certified Examiners [7]

0.2

0.0
0

2

4

6

8

10

0

2

4

6

Defect Height a (mm)Defect Height a (mm)(a) Model-1(b) Model-2

Fig. 6. POD results of (a) Model-1 and (b) Model-2

73

8

10

M. KOJIMA, et al./ Evaluation of Capabilities on Ultrasonic Testing Examiners
Using Probability of Defect Detection and Cumulative Failure Probability

3.3 Cumulative failure probability
Fig. 7 shows the analysis results of cumulative failure probability after 40 years’ operation. Fig. 7(a) shows the results of the Model-1, and Fig. 7 (a) shows the results of the Model-2. The horizontal
axis is the inspection interval (years). The vertical axis is the cumulative failure probability (crack -1).
The green circles are the uncertified examiners’ results of β0. The white circles are the uncertified
examiners’ results of the 95% confidence intervals. The yellow triangles are the certified examiners’
results of β0. The white triangles are the certified examiners’ results of the 95% confidence intervals.
1.E-01Cmulative Failure Probability (crack -1)

Cmulative Failure Probability (crack -1)

1.E-01

1.E-02

1.E-03
UE: Uncetified Examiners
CE: Certified Examiners
95% CI: 95% Confidence Iintervals

1.E-04

UE Eq. (3)UE Eq. (3) ±95%CI
CE Eq. (5)CE Eq. (5) ±95%CI

1.E-05
4

5

6

7

8

9

1.E-02

1.E-03
UE: Uncetified Examiners
CE: Certified Examiners
95% CI: 95% Confidence Iintervals

1.E-04

UE Eq. (4)UE Eq. (7) (: Eq. (4) 95%CI)
CE Eq. (6)CE Eq. (8) (: Eq. (6) 95%CI)

1.E-05
10

4

5

6

7

8

Inspection Interval (years)

Inspection Interval (years)(a) Model-1(b) Model-2

910

Fig. 7. PFM analysis results of (a) Model-1 and (b) Model-2 on cumulative failure probabilities

4.

Discussion

4.1 Evaluation of capabilities using experimental data of UT examiners
Eqs. (3)-(6) showed the superiority of the Model-2; the residual sum of squares of the Model-2
was clearly lower than that of Model-1. In addition, Fig. 6 showed that the Model-2 had narrow rangeof the 95% confidence intervals than that of the Model-1.
The examination results in Table 4(b) showed that the certified examiners detected all defects ofheight equal to or greater than 2 mm. In contrast, even though the defect height of UE008 and UE011
in Table 4(a) was greater than 4 mm, the examination results conducted by the uncertified examiners
were less than 1.0 (100%). Therefore, the regression analysis results in Fig. 6 showed the certified
examiners had better capabilities than the uncertified examiners. However, the authors considered thatthe reliability of capability evaluation in a small difference of experimental data should be necessary.
Thus, the authors attempted an additional evaluation in which the examination results at the defectheight of UE008 and UE011 in Table 4(a) being virtually changed to 1.0 (100%). The original
capabilities of the uncertified examiners were compared with a virtually changed capabilities of
uncertified examiners on condition that POD of UE008 and UE011 are 1.0 (100%).
Eqs. (9) and (10) show the calculation results of the Model-1 and the Model-2 under the virtuallychanged data. Fig. 8(a) shows the regression analysis results of the Model-1 described by the Eqs. (3)and (9). Fig. 8(b) shows the regression analysis results of the Model-2 described by the Eqs. (4) and(10). The red circles are the experimental data of UE008 and UE011 conducted by the uncertified
examiners. The green squares are the virtually changed data of UE008 and UE011. The yellow
diamonds are the experimental data except UE008 and UE011 conducted by the uncertified
examiners.

74

M. KOJIMA, et al./ Evaluation of Capabilities on Ultrasonic Testing Examiners
Using Probability of Defect Detection and Cumulative Failure Probability

Model-1 on condition that POD of UE008 and UE011 are 1.0 (100%)

POD(a)  1  exp a   
 0  1.16,  95%CL  0.683,
se  0.173,

 95%CL  1.49

(9)Se  0.545

Model-2 on condition that POD of UE008 and UE011 are 1.0 (100%)

POD(a )  1  exp  1   2  a 
 10  19.0,  195%CL  N/A,

1

 20  10.5,

 2 95%CL  6.04,

se 1  5.66,

se 2  3.08,

 195%CL  10.7
 2 95% CL  N/A(10)

Se  0.111

The comparison of the Model-1 equations between Eqs. (3) and (9) showed that it was not easy
to decide the superiority of capabilities. The results of regression coefficients, 95% confidence
intervals, standard error, and residual sum of squares were close in the equations. In contrast, thecomparison of the Model-2 equations between Eqs. (4) and (10) showed that it was not difficult todecide the superiority of capabilities. Since the result of residual sum of squares in Eq. (4) was about
twice as large as that in Eq. (10), Eq. (10) has higher capability than Eq. (4). In addition, thecomparison of results of equation between Eqs. (4) and (10) as well as Fig. 8 (b) showed that therange of 95% confidence intervals in Eq. (10) was narrower than that in Eq. (4). Therefore, theadditional evaluation showed the possibility of evaluation on UT examiners’ capabilities underconditions of small difference on experimental data. Consequently, it was shown that the POD modelwith low residual sum of squares made it easier to evaluate the change of UT examiners' capabilities.
1.2

1.2
UE011 (CD)UE011 (CD)1.0

Probability of Defect Detection (―)

Probability of Defect Detection (―)

UE008 (CD)UE011 (OD)
UE008 (OD)

0.8

CD Eq. (9)CD Eq. (9) ±95%CI
OD Eq. (3)OD Eq. (3) ±95%CI
UE008 and UE011 (Changed)
UE008 and UE011 (Original)
Except UE008 and UE011

0.6

0.4

0.2

CD: Changed Data on UE008 and UE011
OD: Original Data on UE008 and UE011
95%CI: 95% Confidence Iintervals

0.0

UE008 (CD)

1.0
UE011 (OD)
UE008 (OD)

0.8

CD Eq. (10)CD Eq. (10) 95%CI
OD Eq. (4)OD Eq. (4) 95%CI
UE008 and UE011 (Changed)
UE008 and UE011 (Original)
Except UE008 and UE011

0.6

0.4

0.2

CD: Changed Data on UE008 and UE011
OD: Original Data on UE008 and UE011
95%CI: 95% Confidence Iintervals

0.0
0

2

4

6

8

10

0

2

4

6

Defect Height a (mm)Defect Height a (mm)(a) Model-1(b) Model-2

8

10

Fig. 8. POD results of (a) Model-1 and (b) Model-2 on the original vs. changed data of the uncertified
examiners

4.2 Evaluation of failure risks of piping using cumulative failure probabilities
Fig. 7 showed the result that the cumulative failure probability increases as the inspection
interval gets longer. The Model-2 had lower cumulative failure probabilities and narrower 95%
confidence intervals than the Model-1 in inspection intervals of 4 and 6 years. However, Eqs. (7) and(8) showed the 95% confidence intervals for the Model-2 were only one side. Therefore, the authors

75

M. KOJIMA, et al./ Evaluation of Capabilities on Ultrasonic Testing Examiners
Using Probability of Defect Detection and Cumulative Failure Probability

considered the 95% confidence intervals on both sides in the Model-2 should be necessary. The
authors attempted additional regression analysis of Mode-2 on condition that β2 is constant.
Here, Eq. (2) is transformed into Eq. (11). Fig. 9 shows a concept of Eq. (11) on condition that β2
is constant. PODs of β10, β1-95%CL, and β1+95%CL are described as logistic curves (A), (B), and (C) in
Fig. 9. The logistic curves (A), (B), and (C) pass through points P0, P-95%CL, and P+95%CL. The points P0,
P-95%CL, and P+95%CL are the inflection points of the logistic curves (A), (B), and (C). Additionally, thegradients of the tangent lines are parallel at the points P0, P-95%CL, and P+95%CL.
Model-2

POD(a)  1  exp  1   2  a 

1

(11)

 1  exp  2  a    1 /  2

1

Probability of Defect Detection (―)1.0(A) : POD of β10(B) : POD of β1+95%CI(C) : POD of β1-95%CI

0.8

0.5

(A)

(B)

P+95%CI

P0(C)

P-95%CI

0.3

0.0

0

0.0

5
- β1
0
β2

-2.5β1+95%CI
β2

7.5- β1-95%CI
β2

10Defect Height a (mm)Fig. 9. Concept of Eq. (11) on condition that β2 is constantEqs. (12) and (13) show the results of regression analysis of the Model-2 of the uncertified
examiners and certified on condition that β2 is constant. Fig. 10a shows the POD results. Fig. 10bshows the results of cumulative failure probability.
Model-2 of the uncertified examiners on condition that β2 is constant

POD(a)  1  exp  2  a    1 /  2 1

 10  19.0,  1 95%CL  19.9,
 2   20  10.5 constant  1 95%CL  17.8(12)

Se95%CL  Se  F  Ve  0.213  4.96  0.0213  0.319
n  nUE  n1  11  1  10
Model-2 of the certified examiners on condition that β2 is constant

POD(a )  1  exp  2  a    1 /  2 1

 10  8.82,  195%CL  10.3,
 2   20  8.93 constant  1 95%CL  7.88

Se95%CL  Se  F  Ve  0.0252  4.84  0.00229  0.0362
n  nCE  n1  12  1  11

76

(13)

M. KOJIMA, et al./ Evaluation of Capabilities on Ultrasonic Testing Examiners
Using Probability of Defect Detection and Cumulative Failure Probability

where, Se95%CL is the residual sum of squares in Eqs. (12) and (13) in β1-95%CL and β1+95%CL. Se is theresidual sum of squares in Eq. (12) and Eq. (13) in β10. Ve is the mean square for error in Eqs. (12)and (13) in β10. F is the Snedecor's F-distribution in Eqs. (12) and (13) in β10. n1 is the number of theregression coefficient in Eqs. (12) and (13); since β2 is as constant as β20, β1 is the only regression
coefficient in Eqs. (12) and (13).
Figs. 7 and 10b showed that the POD model with low residual sum of squares had lessuncertainty on the failure risks of piping. Hence, the superiority of the Model-2 in the cumulative
failure probability result was shown as in the regression analysis result. Furthermore, Figs. 7 and 10b
showed the results that the cumulative failure probabilities of both Model-1 and Model-2 performedby certified examiners in the inspection intervals of over 8 years were almost the identical values
performed by uncertified examiners; therefore, it was shown that UT results performed by certified
examiners might have equivalent failure risks of piping as UT results performed by uncertified
examiners if an inspection interval gets longer.
1.E-01Cmulative Failure Probability (crack -1)

Probability of Defect Detection (―)

1.2

1.0

0.8

UE011
UE008

0.6

UE: Uncetified Examiners
CE: Certified Examiners
95%CI: 95% Confidence Iintervals

0.4

UE Eq. (12) (: Eq. (4))
UE Eq. (12) ±95%CI
CE Eq. (13) (: Eq. (6))
CE Eq. (13) ±95%CI
Uncertified Examiners
Certified Examiners [7]

0.2

2

4

6

8

1.E-03
UE: Uncetified Examiners
CE: Certified Examiners
95% CI: 95% Confidence Iintervals

1.E-04

UE Eq. (12) (: Eq. (4))
UE Eq. (12) ±95%CI
CE Eq. (13) (: Eq. (6))
CE Eq. (13) ±95%CI

1.E-050.0
0

1.E-02

4

10

Defect Height a (mm)

5

6

7

8

9

10

Inspection Interval (years)(a) PODs(b) Cumulative failure probabilities

Fig. 10. Results of (a) PODs and (b) cumulative failure probabilities on condition that β2 is constant5.

Conclusions

The authors evaluated effects on the failure risks of piping using the cumulative failure
probabilities and the POD models considering the difference of the UT examiners' capability. The
conclusions from the evaluation are as follows:(1) The POD model with low residual sum of squares made it easier to evaluate the change of the UT
examiners' capabilities.
(2) The POD model with low residual sum of squares had less uncertainty on the failure risks ofpiping.
(3) UT results performed by certified examiners may have equivalent risks of piping as UT results
performed by uncertified examiners if an inspection interval gets longer.

References[1] The American Society of Mechanical Engineers: "ASME Boiler and Pressure Vessel Code – Rules forInservice Inspection of Nuclear Power Plant Components –", ASME BPVC.XI-2015, pp. 78-124.
[2] H. Machida: "Influence of Non-Detective Examination Performance on Reliability of Pipes Having Stress
Corrosion Cracks", Transactions of the Japan Society of Mechanical Engineers (JSME), Volume 77, Issue
782, pp. 1798-1813 (2011). (in Japanese)77M. KOJIMA, et al./ Evaluation of Capabilities on Ultrasonic Testing Examiners
Using Probability of Defect Detection and Cumulative Failure Probability[3] The Japan Society of Mechanical Engineers: “Code for Nuclear Power Generation Facilities – Rules on
Fitness-for-Service for Nuclear Power Plants –”, JSME S NA1-2008. (in Japanese)[4] H. Shohji, Y. Mizutani, T. Yoshida: "Study of Crack Detection Probability for Maintenance Reliability ofPower Plant (Estimation of Crack Detection Probability on the Damage Probability)", The Japan Society of
Mechanical Engineers (JSME), Volume 83, Issue 845 (2017). (in Japanese)[5] P. G. Heasler, S. R. Doctor: "Piping Inspection Round Robin; 5.0 Relationship Between Detection andCrack Size", U. S. Nuclear Regulatory Commission, NUREG/CR-5068, PNNL-10475, pp. 5.1-5.29(1996).
[6] H. Machida: "Reliability Assessment of PLR Piping Based on Domestic SCC Data", The ASME 2007
Pressure Vessels and Piping Division Conference, San Antonio, Texas, USA, July 22-26, PVP2007-26059(2007).
[7] Japan Nuclear Energy Safety Organization (JNES): "Report on Demonstration Project of Inspection
Technique at Nuclear Power Facility (Confirmation of Accuracy for Defect Detection and Sizing on
Ultrasonic Testing) [Summary Version]" [Translated from Japanese.], 05KIZAIHOU-0001, pp. 658-714(2005). (in Japanese)[8] The Japan Electric Association: "The Ultrasonic Examination Guideline for In-Service Inspection of Light
Water Cooled Nuclear Power Plant Components" [Translated from Japanese.], JEAC 4207-2008. (inJapanese)[9] JIS Z 2345-1: "Standard Test Blocks for Ultrasonic Testing - Part 1: A1 Standard Test Block".
[10] The Japan Electric Association: "The Ultrasonic Examination Guideline for In-Service Inspection of Light
Water Cooled Nuclear Power Plant Components" [Translated from Japanese.], JEAC 4207-2004.
[11] H. W. Bargmann: "Pressure Vessel Integrity in the Presence of Defects: The Probabilistic Prediction in
Spite of Scarce Data", Nuclear Engineering Design 93, pp. 289-294 (1986).
[12] T. Haga: "Statistical Analysis for Pharmaceutical Drug Development - Part 3: Nonlinear Model Revised
Edition" [Translated from Japanese.], Scientist Press Company Limited, pp. 1-24 (2016). (in Japanese)[13] Y. Sakiyama, K. Ohashi, Y. Takahashi: "Application of Nonlinear Regression Model to Sigmoid
Dose-Response Relationship in Pharmacological Studies", Folia Pharmacologica Japonica, Volume 132,
Issue 4, pp. 199-206 (2008). (in Japanese)[14] S. Yoshimura, Y. Kanto: "Probabilistic Fracture Mechanics for Risk-Informed Activities – Fundamentals
and Applications –", Probabilistic Fracture Mechanics Subcommittee, Atomic Energy Research Committee,
The Japan Welding Engineering Society, pp. 1-94 (2017).
[15] H. Itoh, D. Kato, K. Osakabe, H. Nishikawa, K. Onizawa: "User’s Manuals of Probabilistic Fracture
Mechanics Analysis Code for Aged Piping, PASCAL-SP", Reactor Safety Research Unit, Nuclear Safety
Research Center, Japan Atomic Energy Agency, pp. 1-48 (2010). (in Japanese)[16] M. Arakawa, K. Narumi, H. Machida, K. Onizawa: "Benchmark Analysis on the Failure Probability
Assessment of Piping with Stress Corrosion Cracks", The ASME 2011 Pressure Vessels and Piping
Division Conference, Baltimore, Maryland, USA, July 17-21, PVP2011-57498 (2011).
[17] H. Machida, N. Yamashita: "Effect of Crack Detection Performance and Sizing Accuracy on Reliability of
Piping with Stress Corrosion Cracks", The ASME 2008 Pressure Vessels and Piping Division Conference,
Chicago, Illinois, USA, July 27-31, PVP2008-61017 (2008).
[18] Ministry of Economy, Trade and Industry: "Integrity Evaluation Method for Reactor Recirculation Piping(Draft)" [Translated from Japanese.], 10th Conference of Task-Force on Structural Integrity of Nuclear
Power Generation Facility, Sub-Committee on Nuclear Safety and Security, Research Committee on
Natural Resource and Energy, Agency for Natural Resource and Energy, METI, June 15th, Reference
Paper 10-7, pp. 12-21 (2004). (in Japanese)[19] Ministry of Economy, Trade and Industry: "Evaluation Example of Integrity of Primary Loop Recirculation
System Piping Based on the Evaluation Result of the Improved Ultrasonic Testing Accuracy" [Translated
from Japanese.], 9th Conference of Task-Force on Structural Integrity of Nuclear Power Generation
Facility, Sub-committee on Nuclear Safety and Security, Research Committee on Natural Resource andEnergy, Agency for Natural Resource and Energy, METI, September 1st, Conference Material 9-6 (2003).
(in Japanese)[20] M. Nagai, N. Miura, H. Shohji: "Piping Integrity Assessment Utilizing Probability Fracture Mechanics Effect of Probability of Detection Failure Probability -", Central Research Institute of Electric Power
Industry (CRIEPI), CRIEPI Research Report Q16007 (2017). (in Japanese)78