![](media/image1.emf){width="5.780555555555556in"
height="8.564583333333333in"}

![](media/image2.emf){width="6.723611111111111in"
height="9.314583333333333in"}

![](media/image3.emf){width="6.727083333333334in"
height="4.993055555555555in"}

**EXECUTIVE SUMMARY**

The Nortec‑30 Eddyscan inspection system is designed for testing
fastener holes with the fastener in place. The Nortec‑30 Eddyscan
inspection system was evaluated using procedures and test specimens
developed to assess the reliability of eddy‑current inspections for
cracks beneath fasteners. To evaluate its reliability, the system was
used to inspect simulated Boeing lap splice joints. The extent and
location of cracks were unknown to the inspector using the equipment.

Inspection setup was based on a reference standard with an
electro‑discharge machined flaw extending 0.10 inch from the edge of the
countersink for the rivet head. The inspector used a subjective rating
scale of 1, 2, or 3 for all signals interpreted as possible crack
indications. The rating of 3 was used when the inspector was certain of
a reportable flaw indication. The rating of 2 was used to indicate a
reasonable certainty that a call of a flaw being present was correct.
The rating of 1 was used to indicate the perception of a signal; but
under the setup conditions used, there was doubt that the indication
should be reported.

All flaws greater than 0.100 inch in length were detected. The largest
flaw missed using the sure rating of 3 was 0.083 inch in length, whereas
the smallest flaw detected was 0.037 inch in length. Probability of
Detection (PoD) curves fit to this data estimate that a 0.063‑inch flaw
would be detected with 0.90 probability. The false call rate was 0.4
percent. Relaxing the criteria for making a call to include both the
ratings of 3 and 2 resulted in a 1 percent false call rate. With this
relaxed criteria the largest flaw missed was 0.06 inch in length, and
the smallest flaw detected was 0.026 inch in length. The estimated PoD
curve indicates that a 0.055‑inch flaw would be detected with 0.90
probability.

Using the most stringent criterion (rating of 3), the Nortec‑30
inspection achieved better probability of detection than was achieved in
a similar experiment using a sliding probe. At the criterion level
including calls rated 3 and 2, the false\~ll rates for the Nortec‑30
were comparable to that obtained with a sliding probe eddy‑current
inspection, but with an approximate shift in probability of detection
curves Of 0.\~01 inch.

The ability of the Nortec‑30 Eddyscan system to use an \"off‑center\"
signal to give direct feedback on centering the probe was a time‑saver.
Lighting conditions under which the inspections were done would have
forced more time to be spent in visually locating and centering rivets
within a template had a pencil probe procedure been used.

---v---

[1. INTRODUCTION.]{.underline}

The Aging Aircraft Nondestructive Inspection Validation Center (AANC) at
Sandia National Laboratories is charged by the Federal Aviation
Administration to support technology transfer, technology assessment,
and technology validation. A key task of the Center is to establish and
apply a consistent and systematic methodology to assess the reliability
of inspections through laboratory and field experiments.

Under this mandate the Nortec‑30 Eddyscan inspection system was
evaluated using procedures and test specimens originally developed to
assess the reliability of eddy‑current inspections for cracks beneath
fasteners. The Nortec‑30 Eddyscan inspection system is designed for
testing fastener holes with the fastener in place. To evaluate its
reliability, the system was used to inspect simulated Boeing lap splice
joints. The extent and location of cracks in the upper skins were
unknown to the inspector using the equipment. The results of the
experiment are discussed in this report in terms of probability of
detection curves. Comparisons are made to baseline laboratory detection
data obtained using other eddy current equipment.

This report is organized into four sections. Section 1 contains a brief
product description. Section 2 discusses the conditions surrounding the
experiment and the setup procedures followed in performing the
experiment. Results of the inspection are presented in the form of
probability of detection (PoD) curves and relative operating
characteristic (ROC) curves in Section 3. A summary and a discussion of
the results are given in section 4.

[2. PRODUCT DESCRIPTION.]{.underline}

The following brief description of the Nortec‑30 Eddyscan system is
condensed from product description information available from Staveley
Instruments, Inc. This description is meant to reflect the basic
technology and capabilities of the instrument. It is not a complete
description of all the capabilities of the instrument.

The Nortec‑30 Eddyscan System is designed for testing fastener holes
with the fastener in place. It consists of a portable instrument with an
accompanying small hand‑held scanner. The instrument uses pulsed eddy
current techniques and a broadband Hall effect sensor.

The scanner is centered over the fastener during the inspection. The
scanner rotates the Hall effect sensor and the driving coil about the
fastener. Eddy currents are induced by the coil into the test surface.
The induced signal is a sharp‑edged magnetic pulse that provides
excitation for a wide range of frequencies. Due to phase velocity
effects, the lower frequency components propagate through the material
at slower rates than do the higher frequency components.

As a result of the differences in propagation times, effects from
defects that are shallower appear earlier on the detected waveform. A
\"gate\" of a certain width is set to start a certain time after the
start of each pulse. The width and start time of the gate are set by the
user and are determined with test standards. The outputs for a
particular gate are arranged sequentially. If there are no flaws and the
scanner is perfectly centered, the resulting outputs would form a
straight line. The presence of a flaw will alter the signal as the Hall
sensor passes. This will cause a \"bump\" to appear in the baseline
signal. By adjusting the gate parameters and the gain applied to the
signal, the instrument sensitivity can be altered for both flaw depth
and size.

If the Hall sensor is not exactly centered over a circular fastener, the
distance between the edge of the fastener and the sensor will vary
sinusoidally.If the changes in distance are relatively small, the
off‑center probe signal will also be roughly sinusoidal. The frequency
of the sine wave will be the rotation frequency of the scanner. Thus,
when the scanner is close to being centered, the remaining off‑center
signal can be removed by subtracting the appropriate sine wave.

The \"off‑center\" signal is not totally discarded. It is used to aid in
positioning the scanner over the fastener. First the signal is filtered.
Then the phase angle of the peak is used to derive off­center direction
and the amplitude of the signal is used to determine the distance from
the center. This information is then used to drive a cross hair on the
display showing relative positioning.

The instrument has been designed to have three (3) modes during
operation. These are:

1\. Free‑running operation

2\. Search for center

3\. \"Frozen\" analysis display

In the free‑running operation the raw data from the scanner are
displayed on the screen. The off­center data and centering cross hairs
are also displayed. Displays are updated as the scanner moves across a
fastener. Upon pressing the \[TEST\] button on the instrument, the
search for center mode is entered. This mode differs from the
free‑running mode in that the instrument is checking that the cross hair
is less than a certain distance from the origin. \"Close enough\" is
represented by a circle on the cross hair display. The actual off‑center
distance is determined by the gain setting on the centering gate.

The instrument will switch to a frozen analysis display once the cross
hair stays in the circle for about 1 second. It is at this time that the
off‑center data are removed from the signal and the processed data are
shown on the screen. A threshold can be set as a percentage of full
scale. If an inspection results in a signal that exceeds this threshold
then an alarm will alert the operator.

[3. EXPERIMENTAL CONDITIONS.]{.underline}

[3.1 TEST SPECIMENS.]{.underline}

The experiment was conducted on forty‑three small specimens and two
large panels, all with simulated lap splices. Each of the small panels
contains 20 inspection sites. The large panels each contain 102
inspection sites. Thus, 1064 rivets were examined. Flaws were introduced
into the sites as described in the following paragraphs.

The small specimens measure twenty by twenty inches. They contain two
sheets of 0.040‑inch­thick 2024 T‑3 clad aluminum. Flaws were grown in
the top sheet before the sheets were joined as a lap splice. The flaws
were grown by fatigue cycling aluminum plates with undersize holes and
starter cuts at the desired locations. All signs of the starter cuts
were removed when the specimens were drilled to dimensions for joining
as lap splices. See figure 1.

The large panels are eight and one half feet long and approximately four
feet wide. These panels contain a single lap splice joint with all the
frame structure behind the skins that would be in place in a typical
Boeing narrow‑body aircraft such as the 727 or 737. The flaws in these
panels were grown through fatigue cycling. This was done on a special
test bed that simulated the fuselage bi­axial pressure induced stress
typically encountered in one flight cycle. Details about the test
specimen structure can be found in references 1 and 2.

The surfaces of all the small specimens and one of the two large panels
were painted with a typical aircraft paint. The surface of the second
large panel was bare aluminum. A thin transparent 0.003‑inch‑thick tape
was covering the inspection sites on the small specimens. This tape had
been used in prior experiments (reference 2) to protect the test
specimen surface from being scratched by probes and thereby providing
visual clues to subsequent inspectors. However, small air bubbles
beneath the tape were found to be affecting the inspections with the
Nortec‑30. The inspection probe on the Nortec‑30 did not make surface
contact with the specimen during the rotation of the sensors. Therefore,
the tape was removed when it was determined that no adverse marring of
the test specimen surface would occur.

![](media/image4.emf){width="6.723611111111111in"
height="4.7555555555555555in"}

[3.2 INSPECTION EQUIPMENT SETUP.]{.underline}

The inspections using the Nortec‑30 were performed by the Eddy Current
Product Manager from Staveley Instruments, Inc. The inspector set up the
equipment using a Boeing Reference Standard #369 \[3\] supplied by the
AANC. The Boeing #369 Standard contains an electro­discharge machined
(EDM) notch that extends 0.100 inch from the edge ofthe countersink. The
initial setup was checked against a standard in the possession of the
inspector. This second standard had a 0.060‑inch flaw. The setup was
modified slightly to enhance the indication for this smaller flaw.

The resulting setup parameters were:

> Gain 28.5 db
>
> Gate start 100ms
>
> Gate width 30ms
>
> Alarm level not used
>
> Rotation 354 degrees

For all inspections, the inspector was asked to use a 3‑point subjective
rating scale. A rating of 3 would mean that the inspector is certain
that there is a reportable flaw indication. A rating of 2 indicates a
reasonable certainty that a call of a flaw being present is correct. A
rating of 1 is used to indicate the perception of a signal, but some
doubt about the indication actually being reportable. For this
inspection, the rating of 1 was applied to those signals that the
inspector used such descriptions as, \"There seems to be a signal,
although at the current sensitivity level it would probably be passed
by.\"

The inspector gave locations for all flaw indications. A monitor was
present during the inspection and recorded the calls as they were made.
The inspection of the 860 sites (20 x 43) on the small panels was done
in a laboratory on a table top with the equipment operating on standard
110 AC wall current. The inspection of the 204 sites (102 x 2) on the
large panels was done in the hangar with the Nortec‑30 operating from
batteries. The large panels were hanging vertically, simulating the
fuselage of an aircraft.

[4. INSPECTION RESULTS.]{.underline}

The 43 small panels were inspected in a five‑hour period that included
approximately one and one half hours for lunch and other breaks. The
large panels were inspected in fifty minutes. The individual site
(rivet) inspections, in general, took between 6 and 20 seconds, with an
overall average of approximately 14 seconds per inspection site.

[4.1 RELATIVE OPERATING CHARACTERISTICS.]{.underline}

A summary of the results of the inspection is given in table 1. In table
1, flaws are grouped into three categories which correspond to being
under the rivet head (\<0.05 inch), out from the rivet edge but less
than the Boeing setup standard (0.05 to 0.10 inch), and larger than the
Boeing setup standard (\>0.010 inch).

‑4­-

Figure 2 presents the data of table 1 in the form of empirical relative
operating characteristic curves. The points of the curves are the
proportion of detects in each of the categories versus the false call
rate. The first point of each curve is based on the strictest of
criterion levels, that is the rating of 3. The second point shows how
the detection rate varies when the criterion for making a call is
relaxed, as is reflected by considering the 3 and 2 ratings. The third
point reflects the most lenient criteria by incorporating the ratings of
1 into the detection level. The fourth point at (1,1) is the point that
corresponds to calling every point a flaw. All ROCcurves go through the
point (1,1)

![](media/image5.emf){width="6.723611111111111in" height="6.40625in"}

[4.2 PROBABILITY OF DETECTION CURVES.]{.underline}

For the purposes of quantifying the probability of detection, curves are
fit to the results (detect or miss) of the inspection for the known
flaws. Four different mathematical fits are given for the data in
figures 3 through 5. Figure 3 presents the curves fit to the \"sure\"
calls (3\'s). Figure 4 presents the fitted curves for the detects with
ratings of 3 or 2. Similarly, in figure 5 those curves fitting the most
lenient criterion used (l\'s, 2\'s, and 3\'s) are given. False call
rates (FCR)are given as the percentage of unflawed rivets where calls
were made.

![](media/image6.emf){width="6.745833333333334in" height="2.41875in"}

The parameters, a and ,B, were estimated using maximum likelihood
methods applied to binary response data as implemented by the SASS
procedure Probit \[4\]. The regression type models for binary data can
be applied to the occurrence of either event, detect or miss. The
symmetry of the Normal distribution guarantees that the fitted curve for
the probability of detection is independent of whether one fits a curve
to the detects or fits a curve to the probability of misses and then
takes the complement. The parameters, a and ,B, will differ in sign, but
will have the same magnitude. The same is true for the Logistic
distribution. The Gompertz distribution, however, does not have that
symmetry. The result of fitting the Gompertz distribution to the
probability of detection is denoted Gompertz(l). The result of fitting
the same distribution to the probability of nondetection and taking the
complement is denoted as Gompertz(O).

There is no inherent reason why one curve should be adopted over
another. The different forms are given here to illustrate variations
present from the choice of representation. The Logistic and the Normal
curves are very similar and are the most prevalent in use. The Normal
curves for the three different Nortec 30 criterion levels are repeated
in figure 6. They are compared with similar curve fits to laboratory
inspection data gathered using sliding probe and template procedures
\[2\]. The backgrounds on the reference 2 inspections are given in table
2.

All curves presented to this point have been best fit (maximum
likelihood) curves. In figure 7, lower 95% confidence curves are given
for the probabilities of detection for selected curves. All the fits in
figure 7 are for the Normal form of the distribution. The curves not
shown have similar shifts in the lower confidence curve compared to the
curve of best fit.

‑6­-

![](media/image7.emf){width="6.0569444444444445in"
height="9.527777777777779in"}

![](media/image8.emf){width="6.0569444444444445in"
height="9.527777777777779in"}

![](media/image9.emf){width="6.0569444444444445in"
height="9.527777777777779in"}

**[5. SUMMARY AND DISCUSSION.]{.underline}**

The reported inspection results represent a single inspector using the
Nortec‑30 Eddyscan. The inspector was an Eddy Current Product Manager
from Staveley Instruments, Inc. and was well versed with the use of the
instrument.

The inspector gave subjective ratings to the various calls. The
relatively steep receiver operating curve (figure 2) for the class of
flaws less than 0.050 inch in length indicates the presence of signal
above background noises. Half of the flaws detected in this category
were rated as 1. In these cases the inspector expressed the opinion that
a signal was likely present but that the initial setup (on O.100‑inch
flaw) was not sensitive enough to give a clear indication.

All flaws greater than 0.100 inch in length were detected. The largest
flaw missed, smallest flaw detected, 0.90 probability fit, and flaw size
for which the lower 95% confidence interval on probability of detection
exceeds 0.9 for each criteria level are given in table 3. The latter two
values are estimated using the Normal distribution function.

![](media/image10.emf){width="6.738888888888889in"
height="2.3583333333333334in"}

Using the most stringent criterion, the Nortec‑30 inspection achieved
better probability of detection than was achieved using a sliding probe.
At the moderate criterion level, the false call rates for the Nortec‑30
are comparable to that obtained with the sliding probe (EC 1 of figure
6) but with an approximate shift in probability of detection curves of
0.01 inch.

With the most lenient criterion level, the Nortec‑30 probability of
detection curve is almost identical with that fit to the highly
qualified inspectors using template and pencil probe (EC2 and EC3 of
figure 6). However, the false call rate was almost 3% as compared to
0.3% with the template procedures.

The ability of the Nortec‑30 Eddyscan system to use the \"off‑center\"
signal to give direct feedback on centering the probe was a time‑saver.
No appreciable time differences were observed in inspecting the painted
large panel versus the unpainted large panel. However, the lighting
conditions under which the inspections were done would have forced more
time to be spent on the painted panel in visually locating and centering
rivets within a template had a pencil probe

‑10­-

been used. The inspections of 204 rivet sites on panels that simulated
an aircraft fuselage took 53 minutes to complete.

All the Nortec‑30 inspection results are from a single inspector
performing inspections over a large number of test specimens. This
enables probability of detection curves to be fitted to each
inspector‑equipment combination. Inspector‑to‑inspector variations exist
as is evidenced in comparing EC3 and EC4 results. The information on the
Nortec ‑30 Eddyscan System presented here should be taken as reflective
of capabilities when used by a well‑trained inspector using setup
procedures similar to those currently employed in field inspections.

‑ 1 1 ‑

[6. REFERENCES.]{.underline}

1\. Spencer, Borgonovi, Roach, Schurman, and Smith, \"Reliability
Assessment at Airline Inspection Facilities, Vol II.: Protocol for an
Eddy Current Inspection Reliability Experiment,\" DOT/FAA/CT‑92/12, II,
May 93.

2\. Spencer, Floyd and Schurman, Don, \"Reliability Assessment at
Airline Inspection Facilities, Vol III.: Results of an Eddy Current
Inspection Reliability Experiment\" DOT/FAA/CT‑92/12, III, May 95.

3\. Boeing Procedures, NDT Manual, Part 6, 51‑00‑00, April 1992.

4\. SAS Institute Inc., *SAS/STA T. User \'s Guide, Version 6, Fourth
Edition\', Volume* 2, Cary, NC: SAS Institute Inc., 1989.

‑12­