BNL Technical Report

TR-3270-690

MAINTENANCE TEAM INSPECTION RESULTS:

INSIGHTS RELATED TO PLANT AGING

Prepared by-.

W. Gunther

Engineering Technology Division

Department of Nuclear Energy

Brookhaven National Laboratory

June 1990

FIN A-3270

Prepared for the US. Nuclear Regulatory Commission

Office of Nuclear Regulatory Research
CONTENM

Page

1. Introduction. .................................................

2 Background ..................................................

3. Discussion...................................................

Maintenance Program Weaknesses ...................................

Maintenance Program Weaknesses - Summary ...........................

Maintenance Program Strengths ....................................

Summary of Maintenance Program Strengths ............................

Condition Monitoring Techniques ...................................

Summary of Condition Monitoring Techniques ...........................

Maintenance on Specific Components ................................

General Comments .............................................

4. Conclusions and Recommendations ..................................

5. References ..................................................

LIST OF TABLES

1 'Aging' Findings from Selected Maintenance Reports ......................

2 Summary of Maintenance Program Weaknesses ..........................

3 Summary of Maintenance Program Strengths ............................

4 Condition Monitoring Techniques ...................................

1

2

4

4

7

8

9

10

11

12

13

16

17

3

7

9

11

iii
INTRODUCTION

The maintenance team inspections conducted over the past two years have provided the NRC

with an abundance of information about the content and quality of utility maintenance programs.

These in-depth performance based inspections have addressed areas such as:

1) overall plant performance related to maintenance,

2) management support of maintenance, and

3) the implementation of the maintenance program.

Incorporated in the inspection criteria established for these general areas are specific attributes

that are related to the concept of understanding and managing aging. These include:

* root cause evaluation of equipment failure,

* trending of failure data,

* equipment qualification program implementation,

* control of spare parts,

* evaluation of test results, including post maintenance testing, and

* condition monitoring techniques.

The inspection team's judgement of how the above attributes are treated in the utility maintenance

programs reflects on the ability of the plants to manage the effects of aging on safety related

structures, systems and components. Therefore, a selected number of maintenance team inspection

reports were reviewed with the following objectives in mind:

1. Assess the evaluations of those portions of the maintenance program determined to be

important for understanding and managing aging.

2. Evaluate the weaknesses noted in utility maintenance programs which could effect the ability

of the plant to manage aging.

3. Determine the types of preventive maintenance activities and condition monitoring techniques

used which address plant aging.

1
BACKGROUND

The maintenance team inspections have been performed in accordance with temporary instruction

(TI) 2515/97, entitled "Maintenance Inspection". The purpose of this inspection is to determine the

effectiveness of the total integrated maintenance process. As specified in the TI, 'the goal of the

inspection effort is to emphasize the use of plant experience, test and surveillance data, recent

component failures, [and] PRA insights..." to identify strengths and weaknesses. Two-volumes-of

inspection.guidnc-e supplement theTI, and-direct the inspector to evaluate the maintenanc~iagin`;

wassociation. For instance, in section 4.2.1.2 bf the- suploemeftal'guidance, the inspector is directed to

deteriiiiin' the extent to which management is aware of plant aging. Similarly, section 4.22.7 requires

the inspector to evaluate the involvement of corporate management in maintenance activities which

address plant aging.

More important than these citations, however, is the assessment of in crance program activities :,

QWhich' reflect on the plant's ability to manage aging.,Predictive and preventive maintenance programs

must inclu~de& cditifinMonitoring,-trending, andr'ecord keeping in order to effectively manage the

effects of aging. Inadequacies found in these areas are indicative of a plant's inability.to properlyAt this time, 47 inspections have been completed, with 24 more scheduled to be conducted

through the end of 1990. The Performance and Quality Evaluation Branch of NRR has compiled the

findings from these inspections in a computerized database. With their assistance, the plants were

identified where the NRC inspection teams had reported concerns about how the maintenance program

accounted for the effects of aging. Ten reports were obtained which contained specific findings that

the utility maintenance programs do not address aging. These 'aging' findings from the database are

summarized in Table 1.

It should be:noted that nine out of .these ten inspections concluded that the overall maintenance,

Cprogramsweieiidequate,-satisfact-ory, or good. Thus,despit'ethe.fact that deficiencieswere-found in--

the afiagement of aging atihese facilities, their overall maintenanceiprboigramns did adequately address -

sOther. NRC 'concens7 -Based -on -the-guidancekprovided -to -the -mainte'nance in'spectio~n teams, a

maintenance progiram can be judged to be good because it effectively manages current problems, but

rir-ot -effectiiely manage the long term aging -effects 'on components,- systems, and structures.

2
,__Table.1:'..'AgingFmdings fom Slcted Maintenance Reports '

Plant Finding

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

Dresden 2 & 3

Maine Yankee

St. Lucie 1 & 2

Arkansas Nuclear One 1 & 2

Hope Creek

LaSalle 1 & 2

Waterford 3

Grand Gulf 1

Clinton 1

Indian Point 2

Aging of electrical components not considered

No formal program to address plant aging

limited life components not tested in PM Program

Lack of awareness of aging

Aging issue not fully addressed

No program to identify trends in plant aging

No formal program to address plant aging

No program in place to address plant aging affects

No followup of aging failures

Data is not used in support of Plant Aging Program

3
DISCUSSION

Ten of the completed maintenance team inspection reports have been reviewed to obtain the

following information related to aging.

I) weaknesses in the maintenance programs

2) maintenance program strengths

3) applied condition monitoring techniques

4) maintenance activities on specific components, systems and structures

5) NRC inspector concerns/comments

Weaknesses were found in all ten plants in the area of aging. Weaknesses were also found in

areas that the NPAR Program has identified as being important to understanding and managing aging

during both the original license term or for plant life extension. These are listed on a plant by plant

basis and summarized following this listing.

In a similar manner, maintenance program strengths, condition monitoring techniques, and specific

maintenance activities being conducted at the plants as described in the maintenance team inspection

reports are listed and discussed in separate sections. Finally, some general comments and observations

made by the inspection team related to the maintenance/aging management correlation are presented.

These illustrate some maintenance program activities which concerned the inspection team.

Maintenance Programn Weaknesses

Nuclear power plant maintenance programs should include activities which will be effective in

detecting and mitigating the effects of aging. The ten maintenance team inspection reports which are

discussed in this report contain findings indicating that aging is not being properly addressed. These

are presented in this section, along with other findings or observations also indicative of a potential

inability to manage aging.

a. Dresden

* Plant aging consideration for electrical components had not been addressed. (p.44)

* Lack of 'as-found' condition and maintenance cause' was considered a weakness because

trending, rework identification, and root cause analysis would be hindered. (p.13)

* A trend program based on equipment histories had not been established. (p.46)

4
b. Maine Yankee

* No formalized program exists that addresses plant aging considerations. (p.21)

* Engineering weaknesses included the lack of the system engineer concept, no failure trending

analysis involvement, minimum input to corrective and preventive maintenance procedures.

(p.5 )

* Trending is done only on an individual component basis and NPRDS is not used. (p.6)

* 'The lack of incorporation of PRA considerations into the maintenance program is considered

a weakness.' (p25)

c. Arkansas Nuclear One

* The apparent lack of awareness of plant aging is considered a weakness. It appears to be

attributable to an unimplemented equipment trending program.' (p3)

* "Maintenance history, component failure analysis, and trending were determined to be weak."

(p.2 )

* "There was a lack of awareness of plant aging of nonsafety related equipment as evidenced

by numerous problems with the instrument air system." (p.9)

* Electrical, mechanical, and I&C groups have made little progress in identifying equipment data

to be trended. (p.10)

* 'The NRC inspectors found weaknesses in both the licensee's program for root cause analysis

and its implementation." (p.11)

* "The role of probability risk analysis was noted to be nonexistent in the maintenance process."

(p.18)

d. St. Lucie

* Known limited life components are not included in the PM Program - specific reference to

NPAR and electrolytic capacitors. (p.13)

* Vendor recommendations were not properly incorporated into procedures for mechanical

maintenance. (p.64)

* The sensitivity to minor deficiencies needs to be increased. (p.64)

e. Hope Creek

* Plant aging issues have not been fully addressed. (p.12)

* The inspection team was critical of the oil sampling and analysis procedure for the EDGs.

(p.29)

5
f. LaSalle

* Did not have an established program to identify subtle trends in plant aging or common mode

failures in electrical components. (p.18)

* Set point drift problems not adequately evaluated to eliminate root cause. (p.22)

* Diagnostic program for motor-operated valves of a narrow scope and considered to be behind

the industry. (p.5)

g. Waterford

* No formal program had been initiated to address plant aging awareness. (p.15)

* Inadequate attention to ISI and IST results. (p.18)

* She NRC inspector considered the licensee's program for lubricating MOVs inadequate for

preventing the mixing of greases of different soap bases in MOV main gear boxes." (p.14)

* No programmatic mechanism to assure that system engineers were made aware of pertinent

industry operating experience. (p.13)

* Post maintenance testing of compressors not being performed. The capacity of this

equipment was between 50% and 65%. (p.35)

* 'There were few component-specific corrective maintenance procedures for mechanical

components. (p.4)

h. Grand Gulf

* 'A noted weakness in the program was that there was no specific program in place to address

plant component aging...." (p.17)

* ".. maintenance requirements for non-safety related equipment based upon the service life of

the constituent parts of that equipment had not been established." (p.17)

* "Areas identified as weaknesses elsewhere in this report such as root cause analysis,

maintenance, and component trending, ...are in part due to weaknesses in the System

Engineering Group." (p21)

* The use of PRA concepts is non-existent in the GGNS maintenance program." (p.22)

* The description of the problem and the corrective action on the work request were too

cryptic to be useful for equipment history. (p.40)

6
i. Clinton

* "There was no formal method to assess the corrective action for these types of failures to

determine the extent that the operability of other identical components could have been

subjected to common mode 'Plant Aging' failure." (p.15)

* The inspectors concluded that the system engineers' involvement was weak in the MWR

process, root cause analysis, field system and component walkdowns..." (p.15)

* The licensee had not established goals for measuring the effectiveness of maintenance. (p5)

j. Indian Point 2

* Management has no effective indicator available to monitor plant aging or the effectiveness

of site maintenance to impact plant aging. (p.11)

* The licensee's maintenance program also lacks a thorough incorporation of vendor

maintenance recommendations into the preventive maintenance program." (p.14)

* "While environmental (EQ) required maintenance was defined, a program with procedures

to implement, schedule, and track this EQ required maintenance was not yet in place." (p.16)

* A systematic method does not exist for defining equipment maintenance requirements for

inclusion in the PM program. (p.16)

Maintenance Program Weakness - Summary

The maintenance program weaknesses which were identified in more than one of the inspection

reports are summarized in Table 2. Each of these relates to some degree to the ability of the licensee

to manage the effects of aging.

Table 2: Summary of Maintenance Program Weakncsscs

Observation # of Plants

Management of Aging Not Addressed 10

(see Table 1 for details)

Inadequate trending 5

Inadequate root cause analysis 4

System engineer program deficiency 4

PRA considerations not included 3

Maintenance history not maintained 3

Vendor recommendations not considered 2

No method to identify PM requirements 2

Insufficient diagnostic maintenance 2

7
Maintenance ProEgram Strcnntbs

In the course of the maintenance team inspections, the inspectors recorded positive observations

in the programmatic and equipment-specific aspects of the utility maintenance programs. Those aspects

necessary for managing aging which were documented in the reports have been extracted and are

discussed below. Reference citations pertain to the page numbers of the applicable inspection report.

a. Dresden

* Maintains a setpoint trending program for bistables. (p.21)

c Employs a time series analysis of equipmeni failures to determine preventive maintenance

requirements. (p.43)

b. Maine Yankee

* Service life information is being gathered on safety related electrical equipment and associated

parts; technical efforts are being conducted on mechanical and electrical issues which affect

plant aging. (p.21)

* Licensee has an aggressive work schedule and program for motor operated valves. (p.61)

c. Arkansas Nuclear One

* The preventive maintenance improvement project was determined to be a definite strength.

The entire PM concept had been reevaluated and restructured from new procedures to

implementation. (p.2)

* Motor operated valve diagnostic testing is significantly greater than the industry norm. (p.2)

d. St. Lucie

* All of the instruments used to obtain data for operator logs are included in the calibration

program. (p.54)

* Based on a circuit breaker failure analysis, PM was initiated on all similar breakers in the

control rod drive system. (p.49)

e. Hope Creek

* Numerous performance indicators are available for management to assess the status and

effectiveness of maintenance. (p.4)

* The maintenance management information system (MMIS) is used by system engineers to

analyze completed corrective maintenance to determine any action requirements and inputs

to the station performance trending/monitoring program. (p.24)

8
f. LaSalle

* Has implemented a Problem Assessment Data Sheet (PADS) program to facilitate the

systematic root cause and failure analysis of systems and components. (p.28)

* Trends of results were evident for predictive maintenance in the areas of vibration data and

lube oil samples. (p30)

g. Waterford

* Has applied lessons learned of past equipment problems and repairs to PM Program to

enhance reliability and availability. (p31)

* A root cause evaluation of all failures is conducted with a view toward extending the

operating life of a component, or quantifying failure information in order to better predict

when a component will fail again. (p32)

* All maintenance activities on both safety related and BOP equipment were stored in the

computer data base and in the NPRDS; the computer data base appeared to be an excellent

basis for maintenance trending. (p31)

h. Grand Gulf

* The Independent Safety Evaluation Group (ISEG) reviews plant operations and maintenance

experiences; evaluating and trending those experiences, reporting to NPRDS, and making

recommendations to plant management. (p.21)

* The Station Information Management System is a tool that will improve component trending

analysis. (p. 29)

Summary of Maintenance Program Strengeths

Table 3 summarizes these maintenance program strengths identified in the maintenance team

inspection reports which could be incorporated into a program to address plant aging.

Table 3: Summary of Maintenance Program Strengths

Observation # of Plants

Evaluation of failures for further action 6

Trend program in effect 4

Gathering service life data 2

Effective PM program 2

Aggressive MOV program 2

9
Condition Monitoring Techniques

As part of the review of the maintenance team inspections, information about the condition

monitoring techniques that were observed were recorded. These are listed below along with the page

reference from the applicable report.

a. Dresden

* Detection of loose connections by using thermography. (p.15)

* Vibration analysis and lube oil analysis used. (p.15)

* MOV diagnostic performed using 'VOTES", Valve Operator Testing and Evaluation System.

(p.18)

* Sonic test equipment used to detect instrument air leaks. (p.15)

b. Maine Yankee

* MOV diagnostic testing performed using MOVATs. (p.61)

c. Arkansas Nuclear One

* MOV diagnostic tests (MOVATs). (p.2)

* Vibration and loose parts monitoring system; however, found by the inspection team to be

inoperable. (p.17)

d. St. Lucie

* "The licensee has had several successes in both the vibration and thermography areas." (p.28)

* The cause of unplanned shutdowns are analyzed - critical system parameters are then

monitored and trended to prevent future shutdowns. (p31)

e. Hope Creek

* Calibration data is evaluated and noise analysis conducted for pressure transmitters. (p.32)

£ LaSalle

* The electrical maintenance monitoring included vibration analysis of motors, analysis of EDG

lubrication oil, and thermography. (p.16)

* Oil samples from electric motors and transformers were analyzed, but the results were not

trended. (p.18)

* Performance monitoring of heat exchangers and erosion/corrosion pipe monitoring in early

stages of implementation. (p.4)

10
g. Waterford

* All safety related and most BOP mechanical components are included in the vibration

monitoring program. (p.36)

* Items trended include flow rates, pressures, temperatures, and instrument setpoint drift.

(p.36)

h. Grand Gulf

* Trending of motor megger readings, specific gravity for batteries, lube oil analysis, and

vibration. (p34)

i. Clinton

* Electrical maintenance included monitoring techniques such as vibration analysis, motor oil

samples, and thermography to detect loose electrical terminations. (p.7)

j. Indian Point 2

* The licensee has implemented several commendable efforts at predictive maintenance

including the use of thermography to find problem spots and valve stroke time trending to

trigger preventive maintenance. (p.16)

Summary of Condition Monitoring Techniques

An important aspect in any maintenance program is the ability to assess the condition of the

equipment. In each of the inspection reports, the team provided examples of the predictive

maintenance practices. That is, those parameters that were being monitored in order to detect

degradation of the equipment or system prior to a failure are described. Table 4 lists the techniques

highlighted in the inspection reports.

Table 4: Condition Monitoring Techniques

Technique # of Plants

Vibration analysis 7

Thermography 5

Lube oil analysis 4

MOV diagnostics 4

System parameters 2

Noise analysis, erosion/corrosion monitoring. 1

insulation resistance of motors, and specific

gravity of batteries

11
Maintenance on Specific Components

Several of the maintenance team inspection reports described maintenance activities or a lack of

maintenance for specific components which were evaluated or observed. Of particular interest here

are citations associated with components or systems studied in the NPAR Program.

a. Dresden - None specifically identified.

b. Maine Yankee

* Cleaning and inspection of motor control centers. (p35)

* Refurbishment of 83 General Electric AK-25, 480 volt circuit breakers. (p.37)

* Cleaning and inspection of battery chargers and inverters. (p.38)

* Upgrade of power semiconductors and cable for battery chargers. (p.38)

c. Arkansas Nuclear One - None specifically identified.

d. St. Lucie

* Lack of PM on molded case circuit breakers.

* PM Program to inspect and test capacitors in inverters and battery chargers - specific

reference to NPAR. (p.13)

* No PM of instrument air filter regulators - changed when they become clogged. (p.15)

* Replaced station battery recently due to 'copper contamination'. (p.11)

* Discrepant condition involved peeling tape on electrical splices. (p.11)

* Significant fuse and fuse holder degradation due to corrosion. 'Since the plants are located

in a salt air environment, the team was concerned that corrosion in the electrical circuits

could be a potential problem." (p.49)

* Control rod drive system molded case circuit breakers were placed in the PM Program due

to setpoint problems. (p.49)

e. Hope Creec

* Performs regular insulation and winding resistance tests on transformers. (p.30)

* EDG generator end PM involves checking the condition of the coils, rotor-stator air gap,

collector rings, and brush rigging. (p.30)

12
f. LaSalle

* EDG operational parameters trended. (p.18)

* Melamine torque switches used in Limitorque MOVs failed due to 'post mold shrinkage,

which was affected by temperature and age. (p.29)

* Upgrade of SOVs on the EDG fuel oil system included the utilization of improved internals

such as Viton '0' rings instead of EPDM 'O' rings. (p.20)

g. Waterford

* PM on 4160 volt switchgear requires torquing of all limit switches, auxiliary switches and

switch tie-bolts. (p.29)

* Reactor trip breakers (General Electric AK-2-25) are returned to GE every five years for

refurbishment. (p.27)

* 'Components of the operators for air operated valves were not in a preventive maintenance

program." (p5)

h. Grand Gulf - None specifically identified.

i. Clinton

* Nitrogen pressure was not being maintained on the electrical penetrations. (p.23)

j. Indian Point 2

* Capacitors are periodically replaced on the Foxboro rack equipment. (p.16)

* An 18 month PM inspection is performed by Westinghouse and plant personnel on the

reactor trip breakers (DB-SOs). (p33)

General Comments

In each of the inspection reports, observations are made by the team which provide insights into

the philosophy of the maintenance program from NRC's perspective. Those that relate to aging are

summarized belowa. Dresden

* An 'alert level' of 2 corrective work requests on a component in 6 months had been

established. The inspection team was concerned that this does not identify trends common

to a specific model number, or longer term trends. (p.25)

13
b. Maine Yankee

* The NRC inspection team suggests a more active use of NPRDS to identify trends and assist

in failure analysis. They also indicated that coordination of MOV work is critical due to

multiple tasks. The NRC inspection team noted that prior to 1987, PM work items had more

deferrals than presently, indicating that PM was not a high priority. (p.65)

c. Arkansas Nuclear One

* The inspectors noted that several hundred engineering activity requests are being evaluated

by the utility for substitution of components no longer manufactured or available. (p.18)

d. St. Lucie

* The team does not consider the ASME Section XI vibrational surveillance a substitute for

vibrational analysis performed as part of a good PM Program. (p38)

* Components with 4 or more failures in 12 months are identified; components from the

NPRDS are identified when their failure rate is 1.6 times the industry average. (p30)

e. Hope Creek

* Periodic reports are obtained from NPRDS comparing failure data for HCGS with industry

averages. A detailed failure analysis is performed if a high rate is obtained - wsignificance

factor". (p.10)

* The reliability centered maintenance program has been formulated for Hope Creek. When

implemented, it will provide a more formal means of addressing plant aging issues and the

decision processes relating to maintaining, upgrading, or replacing equipment. (p.14)

f. LaSalle

* Measure of maintenance effectiveness included backlog and PM/CM ratio. Did not include

number of LCOs or power reductions.

* Trend definition: 3 failures of component in a 12 month period. Same for all components

and does not consider safety impact. (p.30)

g. Waterford

* The licensee maintains a history file for all components, and conducts a root cause evaluation

of all failures. This is done with a view toward extending the operating life of a component,

and quantifying failure information in order to better predict when a component will fail

again. (p32)

* The licensee plans to further improve the trending and predictive maintenance program by

consolidating the various trending systems into one program, and use of enhanced trending

software. (p.36)

14
h. Grand Gulf

* Most PM requirements were generated from startup testing. Systems and components did

not receive equal treatment. (p.18)

Clinton

* 'Aging' was listed as the root cause for 312 safety related equipment failures in last two years.

This was questioned by the inspection team since Clinton is a relatively new plant. (p.15)

j. Indian Point 2

* The licensee is planning to develop a 'Daily On-line Retrieval Information System - DORIS.

to maintain a running record of data obtained from logs, PM, and surveillance test results.

'...the inspection team considers that implementation of a DORIS type program... could

significantly enhance development of the predictive maintenance program." (p.3 5 )

15
CONCLUSION AND REOMMENDATJONS

The ten reports which have been reviewed provided information on how utility maintenance

programs are addressing the aging of components, systems, and structures. (this infor iicludesi

the attitude of management towards the aging issue as well as the status of maintenance program

ittrnbuteswhich addres"'ihe detection or mitigatioii of dradation-caused by'aging. Those attributes

which were of inteiirstto this review include failure root cause analysis, trending of data, and condition

monitoring techniques.

Ad.In general, the findings of the NRC inspection teams indicate that ag jis not being adequately 6

addressed in the main'tenance'programs at the ten plants which 'ere reviewed. Most of these plants

have, or are incorporating, more sophisticated techniques and toolsiifitheir'maintenance programs

which can be used in an effective aging management program. However, the aging issue is apparently

not a high priority, and therefore is not receiving sufficient attention.

On the regulatory side, it is important to note that in each of the ten reports, inadequacies were

reported related to the management of aging. However, in nine of the ten cases, the overall

maintenance program was considered to be 'good', 'satisfactory', or 'effective'.

Because of the insights learned from this limited review, all of the maintenance team inspection

reports are being systematically reviewed. A matrix of practices related to detecting and mitigating the

effects of aging is being developed, along with a more detailed evaluation of utility practices and

attitudes towards the managing of aging.

16
IREFERENCES

1. Report 50-237/88029(DRS), 50-249/88030(DRS); Dresden Nuclear Power Station - Units 2 and

3 Maintenance Team Inspection Report, 4/4/89.

2. Report 50461/89003; Clinton Power Station Maintenance Team Inspection Report, 5/10/89.

3. Report 50-309188-80; Maine Yankee Atomic Power Company Maintenance Team Inspection

Report, 1/18/89.

4. Report 50-313188-36. 50-368/88-36; Arkansas Nuclear One Maintenance Team Inspection Report,

2/7/89.

S. Report 50-335/89-24 and 50-389/89-24; St. Lucie 1 and 2 Maintenance Team Inspection Report,

1/10/90.

6. Report 50-354/89-80; Hope Creek Generating Station Maintenance Team Inspection Report,

1/31/90.

7. Report 50-373/89010(DRS), 50-374/89010(DRS); LaSalle County Station, Units 1 and 2

Maintenance Team Inspection Report, 6/27/89.

8. Report 50-416/88-21; Grand Gulf Maintenance Team Inspection Report, 1/4/88.

9. Report 50-382/89-01; Waterford Steam Electric Station, Unit No3 Maintenance Team Inspection

Report, 3/14/89.

10. Report 50-247/89-80; Indian Point Station Unit No.2 Maintenance Team Inspection Report,

8120/89.

17