U. S. NUCLEAR REGULATORY COMMISSION

OBSERVATION AUDIT REPORT NO. 91-1 

FOR THE INTERNAL AUDIT NO. 90-1-01 OF THE OFFICE OF 

CIVILIAN NUCLEAR WASTE MANAGEMENT HEADQUARTERS AND 

YUCCA MOUNTAIN PROJECT OFFICE

Jakes T. Conway 

"•eository Licensing aa Quality 

Assurance Project Directorate 

.Division of High-Level Waste Management 

k , &H, 2/21/91 

Jo n T. Buckley 

Re pository Licensing and Quality 

Assurance Project Directorate 

Division of High-Level Waste Management

ruce Mabreito (Liytlecon) 

center for Nuce rWaste 

Regulatory Analyses 

Kobert D. Biient tyI'telecon) 

Center for Nuc lea il-aste 

Regulatory Analyses

•illiam el ke 

/Repoiory Licensing and Quality 

/ oitrAssurance Project Directorate 

Division of High-Level Waste Management

Reviewed and Approved by:

'ng&egR.Hooks 0L!9 

Repository Licensing and Quality 

Assurance Project Directorate 

Division of High-Level Waste Management

FULL TEXT ASCII SCAN4 9104020171 910315 

PDR WASTE 

WM-1I PDR
1.0 INTRODUCTION

From October 15-19 and October 22-26, 1990, members of the U.S. Nuclear 

Regulatory Commission (NRC) staff participated as observers on the U.S. 

Department of Energy (DOE) Office of Civilian Nuclear Waste Management 

(OCRWM) Headquarters (HQ)/Yucca MountainWProject Office (YMPO) Quality 

Assurance (QA) Internal Audit No. 90-1-01 of HQ in Washington, D.C. and 

YMPO in Las Vegas, Nevada. 

This report addresses the adequacy and effectiveness of the OCRWM QA 

program as demonstrated by the HQ/YMPO audit team and of implementation 

of the OCRWM QA program by HQ and YMPO technical and QA staff. 

2.0 OBJECTIVES 

The objective of the HQ/YMPO internal audit was to evaluate the 

implementation and effectiveness of the OCRWM QA program in meeting the 

applicable requirements of DOE/RW-0214, Quality Assurance Requirements 

Document (QARD), Rev. 3 and DOE/RW-0215, Quality Assurance Program 

Description, (QAPD), Rev. 3. The NRC staff's objective was to gain 

confidence that HQ and YMPO are properly implementing the requirements of 

the OCRWM QA program in accordance with the QARD, QAPD, and 10 CFR Part 50, 

Appendix B. 

3.0 SUMMARY AND CONCLUSIONS 

The NRC staff based its evaluation of the HQ/YMPO audit process and the 

OCRWM QA program on direct observations of the auditors, discussions with 

the audit team, and reviews of the pertinent audit information (e.g., 

audit plan, checklists, HQ and YMPO documents). The audit was conducted 

in a professional manner, and the programmatic and technical portions of 

the audit were generally effective and well integrated. The audit team 

was well qualified in the QA discipline, and their assignment and checklist 

items were adequately described in the audit plan. 

The NRC staff agrees with the preliminary finding of the audit team that 

the OCRWM QA program has adequate procedural controls in place for the 

areas that were audited. However, the number of areas in which the HQ/YMPO 

audit team identified the OCRWM QA program as ineffective or indeterminate 

is of concern to the NRC staff, particularly the areas of audits and 

corrective actions at HQ and the technical baseline documents at both HQ 

and YMPO. The NRC staff fully supports the audit team's recommendations 

of actions to be taken to verify the effectiveness of corrective actions 

prior to the start of any new site characterization activities. 

OCRWM management must closely monitor HQ and YMPO implementation of the 

OCRWM program to ensure that future implementation is carried out in an 

FUtL EXT ASCII SCAN
K>ý

2 

adequate manner. The NRC staff expects to participate in this monitoring 

as observers and may perform its own audits of HQ and YMPO at a later date 

to independently determine the adequacy and effectiveness of the QA program. 

4.0 AUDIT PARTICIPANTS 

4.1 NRC

Kenneth R. Hooks 

William L. Belke 

James T. Conway 

John T. Buckley 

Bruce Mabrito 

Robert D. Brient 

4.2 DOE 

James Blaylock 

Stephen R. Dana 

Martha J. Mitchell 

Charles C. Warren 

Amelia I. Arceo 

Paul Bryant 

Robert Clark 

A. Edward Cocoros 

Robert B. Constable 

Neil Cox 

Mario Diaz 

James J. George 

William Haslebacher 

John S. Martin 

Marc J. Meyer 

Arthur W. Spooner 

Richard Weeks 

Ardell M. Whiteside 

4.3 State of Nevada

Susan Zimmerman

Observation Team Leader 

Observer (HQ only) 

Observer (YMPO only) 

Observer (HQ only) 

Observer (Center for Nuclear Waste 

Regulatory Analyses - YMPO only) 

Observer (Center for Nuclear Waste 

Regulatory Analyses - HQ only)

Audit Team Manager 

Audit Team Leader 

Lead Technical Specialist 

Lead Auditor 

Auditor (YMPO only) 

Technical Specialist (HQ 

Auditor (HQ only) 

Auditor 

Auditor (YMPO only) 

Auditor 

Auditor 

Auditor 

Technical Specialist 

Auditor 

Technical Specialist 

Auditor 

Auditor (YMPO only) 

Auditor

only)

Observer

4.4 Clark County, Nevada

Engelbrecht von Tiesenhausen 

4.5 Nye County, Nevada 

Phillip A. Niedzielski-Eichner

DOE YMPO 

SAIC 

SAIC 

MACTEC 

SAIC 

SAIC 

DOE HQ 

MACTEC 

DOE YMPO 

SAIC 

DOE YMPO 

CER 

WESTON 

SAIC 

CER 

WESTON 

SAIC 

SAIC

Observer 

Observer

FULL TEXT ASPH SOAN

It 11
3

4.6 Edison Electric Institute 

Thomas Colandrea Observer 

4.7 TRW 

R. James Brackett Observer 

5.0 REVIEW OF THE AUDIT AND AUDITED ORGANIZATION 

The OCRWM internal audit was conducted in accordance with OCRWM QA 

Administrative Procedure (QAAP) 18.2 "Audit Program," Revision 1 

(effective October 15, 1990), and OCRWM QAAP 16.1 "Corrective Action 

Requests," Revision 2 (effective October 15, 1990). 

The NRC staff observation of the HQ/YMPO audit was based on the NRC 

procedure "Conduct of Observation Audits" issued October 6, 1989. NRC 

observer findings are classified in accordance with this procedure. Levels 

1, 2, and 3 of NRC Observations require a written response from DOE to be 

resolved. The NRC findings may also include weaknesses (actions or items 

which are not deficiencies but could be improved), good practices (actions 

or items which enhance the QA program) and requests for information required 

to determine if an action or item is deficient. Written responses to 

weaknesses identified by the NRC staff will be requested when appropriate. 

In general, weaknesses and items related to requests for information will 

be examined by the NRC staff in future audits or surveillances. 

5.1 Purpose/Scope of Audit 

The purpose of the audit was to evaluate the implementation and 

effectiveness of the QA controls applied to OCRWM activities affecting quality. The scope of the audit included those HQ and YMPO activities 

associated with new site characterization, particularly the program level 

technical baseline documents. 

(a) Programmatic Elements 

The programmatic portion of the audit utilized checklists based on 

the requirements in the QAPD and other applicable documents. The 

checklists covered QA program controls for fourteen of the eighteen 

10 CFR Part 50 Appendix B criteria (fifteen of twenty QAPD elements). 

Criteria IX, X, XI, and XIV of 10 CFR Part 50, Appendix B (Sections 

9, 10, 11, 14, and 19 of the QAPD) were not included in the scope of 

the audit since OCRWM currently is not performing activities in these 

areas. 

pULj.rXT ASCAI SCAt4
4

(b) Technical Areas 

Technical activities engaged in by OCRWM were limited. The audit team 

technical specialists were instructed to include the following areas 

in their evaluations: 

(1) qualifications of technical personnel; 

(2) understanding of procedural requirements as they 

pertain to technical activities; 

(3) adequacy of technical plans and procedures; and 

(4) development of study plans and any related work 

products. 

NRC technical staff were not included on the NRC observation team, 

due to the limited technical scope of the audit and, therefore, the 

NRC did not evaluate the technical adequacy of any technical products. 

5.2 Timing of the Audit 

The NRC staff believes the timing of the QA audit was less than optimal, 

since many of the procedures governing both the conduct of the audit and 

the conduct of the QA program activities being audited had been very 

recently issued. For example, the Master List of Controlled Documents and 

the checklists based on Revision 2 of the QARD were replaced after the 

Audit Books were issued to the observers, and the QA Controls Document had 

not been issued by October 19, 1990. However, the NRC staff believes that 

an audit to establish a baseline for the OCRWM program and HQ and YMPO 

implementation was necessary. 

5.3 Examination of Programmatic Elements 

The HQ/YMPO programmatic checklists covered the QA program controls for 

the fifteen elements listed below: 

1.0 Organization 

2.0 Quality Assurance Program 

3.0 Design Control 

4.0 Procurement Document Control 

5.0 Instructions, Plans, Procedures and Drawings 

6.0 Document Control 

7.0 Control of Purchased Items and Services 

8.0 Identification and Control of Materials, Parts, 

Components, and Samples (YMPO only) 

12.0 Control of Measuring and Test Equipment (YMPO only) 

13.0 Handling, Storage, and Shipping (YMPO only) 

15.0 Control of Nonconforming Conditions

FULL TEXT ASCII SCAN
5

16.0 Corrective Action 

17.0 Quality Assurance Records 

18.0 Audits 

20.0 Scientific Investigation Control 

The NRC staff observed the audit team's evaluation of selected 

programmatic elements of the QAPD. Only those elements of the QA program 

which were observed will be addressed in this report. 

(a) Organization (Criterion 1) 

HQ 

The auditors utilized the published audit checklists and were generally 

thorough in reviewing the associated objective evidence. The auditors 

interviewed the Director and Deputy Director of OCRWM, and the Director of 

the OCRWM Office of QA. At times, the auditors appeared to spend an unusual 

amount of their time probing for the desired objective evidence as opposed 

to obtaining it from direct questioning from the checklist. This could 

have possibly been due to the recent OCRWM reorganization (the "interim" 

organization in effect at the time of the audit was implemented July 16, 

1990) and certain newly revised procedures (which appeared to frequently 

contain conflicting requirements). The problems of audit team focus and 

the HQ organizational and procedural changes caused the audit process to 

be lengthened somewhat; however, the desired objectives of the audit were 

eventually achieved. 

The NRC staff identified two areas of concern with the HQ organization. 

First, Section 1, paragraph 1.1.1.(h) of the QAPD requires an annual 

assessment of the scope, status, adequacy, and compliance of the QA program 

by management who are independent of the Office of QA. To date, this 

assessment has not been performed and is tentatively scheduled for June 

1991. The NRC staff believes that this assessment should be scheduled 

earlier since management assessments provide valuable insight in determining 

major problems and in allocating adequate resources for the project. 

Second, the NRC staff noticed that the HQ QA Division appeared to be 

understaffed, with only two full-time DOE personnel reporting to the QA 

Division Director. The NRC staff submitted an Audit Observer Inquiry form 

noting that the QA Division appeared to be understaffed and requested an 

explanation of how the OCRWM determines whether sufficient resources have 

been allocated for the HQ QA Division to accomplish its mission. The DOE 

response to this inquiry indicated that there are currently two positions 

posted and waiting to be filled. In addition, with the number of contract 

support personnel available, OCRWM believes QA staffing should be adequate 

for future activities. The NRC staff agrees with this response. 

JFULL TEXT ASCII SCAN
6 

YMPO 

The auditors used the published audit checklist which consisted of twelve 

requirements from the QARD document. The NRC staff noted that checklist 

items were not taken from Quality Management Procedure (QMP)-01-01 

"Organization." During the audit, two additional requirements from QMP-01-01 

were added to the checklist. The auditors interviewed the Director - QA 

Division; Director and Deputy Director - Regulatory and Site Evaluation 

Division (RSED); and Branch Chief - Site Investigations Branch to verify 

the requirements identified on the checklist for Criterion 1. The auditors 

seemed to have a problem differentiating between the responsibilities of the 

OCRWM Director, Office of QA and the YMPO Director - QA Division, both 

positions of which are currently filled by the same individual. The 

auditors asked specific questions from the checklist, but in many cases, 

answered their own question before the auditees could give a response. 

The NRC staff believes that the auditors could have been better prepared, 

but they satisfactorily completed the audit checklist and evaluated the 

YMPO organizational structure for compliance to the QAPD. 

(b) Quality Assurance Program (Criterion 2) 

HQ 

The auditors reviewed and evaluated the adequacy and effectiveness of 

personnel qualifications for the OCRWM Office of Systems and Compliance. 

Evaluations of the personnel indoctrination and training matrix, education, 

experience, position descriptions, and engineering series were performed. 

In addition, the required annual supervisory evaluations of the employee 

positions were reviewed to assure appropriateness for the work being 

performed by the employees. 

The auditors used the checklist questions and probed beyond the checklist 

questions in sufficient detail to accomplish a comprehensive audit. As 

the auditors proceeded through the checklist and related questions, the 

auditors explained to the auditees and observers exactly what was being 

requested, why it was being requested, and whether the information received 

was acceptable. This type of auditing style facilitated the audit process 

and was an effective method for expediting the audit. The desired information 

was produced in a timely manner and questions and clarifications from the 

auditee and observers were minimal. 

Personnel qualification records were complete, accurate, and thoroughly 

documented. Prior Privacy Act restrictions limiting access to personnel 

qualification files during this phase of the audit imposed no problems. 

All personnel qualification files in this area were open and readily 

available for the auditing process. Information in the personnel 

qualification files was sufficiently detailed and accurately reflected the 

education, experience, and specific responsibilities required for the 

particular position. The indoctrination and training matrix described the 

required training and the date the individual employee received it. 

Documentation also indicated that each individual employee had been evaluated 

FULL TEXT ASCII SCAN
7 

by management on an annual basis to verify the accuracy and appropriateness 

of the work currently being performed by the employee. However, the NRC 

staff questioned how training or retraining is accomplished when new or 

revised procedures are issued. This question was not included in the 

checklist and appears to be an open issue. 

The auditors and auditees were well prepared and knowledgeable of the 

subject matter in the area being audited. However, the apparent lack of 

familiarity with procedural requirements displayed by some HQ staff during 

other portions of the audit suggests that training may not be particularly 

effective. Consequently, for this element of the audit, the QA Program is 

adequate and, with the possible exception of QA training, is being 

effectively implemented. The audit under this criterion was effective. 

YMPO 

The NRC staff evaluated the audit portion of Criterion 2 that pertained 

only to training and qualifications of personnel at the YMPO. Procedure 

QMP-02-01 establishes the requirements for implementing the qualification 

evaluation process; defining the indoctrination and training program; and 

documenting qualification, indoctrination, and training activities. The 

procedure is applicable to the DOE, MACTEC, and Science Applications 

International Corporation (SAIC) Technical and Management Support Services 

(T&MSS) personnel who perform quality related activities under the OCRWM 

QAPD. 

Under the qualification process, the individual's manager and personnel 

from the Human Resources Department of SAIC compare the Position 

Description (PD) with the individual's accumulated skills, training, and 

experience. In the indoctrination and training program, the individual's 

manager determines the training needs and is responsible for assuring that 

each individual receives and completes the assigned training. The development 

and conduct of training is covered in procedure QMP-02-09. The two levels 

of training consist of indoctrination (reading assignments and orientation 

briefings) and training/proficiency (in-depth instruction). 

The auditors' evaluation of personnel qualification, indoctrination and 

training included a review of personnel records and interviews with training 

supervisors from DOE, SAIC, and MACTEC. The T&MSS group of SAIC is responsible 

for the training of personnel at the YMPO and also for the maintenance of 

their records. 

The auditors sampled the training file by selecting a number of individuals 

from the YMPO organizational charts of DOE and MACTEC. Approximately 18 

DOE training files were reviewed. The auditors reviewed six MACTEC training 

files which consisted of PDs, Qualification and Proficiency Evaluation Forms, 

Resumes, Copies of Certifications, and Training and Reading Assignments. 

The auditors were told that the T&MSS training files are similar to DOE's, 

and they will be reviewed during the audit of SAIC the week of November 12, 

1990. All the files that were reviewed appeared to be complete, and 

each individual had completed the required reading assignments prior to 

FULL TEXT ASCII SCAN

It . 1
I'll,

8 

performing quality affecting activities. It was noted by the NRC observers 

that the Proficiency Evaluation Form was deleted in an Interim Change Notice 

dated October 19, 1990, to QMP-02-O1. Thus, management's annual or semi-annual 

evaluation of an individual's performance will not be performed in the 

future. 

The auditors reviewed the certification records for those individuals who 

have been performing OCRWM external audits of the participant QA programs. 

Unlike the other training files retained by SAIC, the auditor certification 

records are maintained by the YMPO QA Division. These files consist of 

Record of Auditor/Lead Auditor Qualification, Training Assignment, Resume, 

Audit Participation Record, and a Qualification Statement. Approximately 

31 files were reviewed by the auditor, and they were found to be 

satisfactory. 

Based on the depth of the evaluation and the completion of the applicable 

checklist items, the audit of training and qualification records appeared 

to be effective, and the implementation by DOE/YMPO appeared to be 

adequate. 

(c) Design Control (Criteria 3) 

The audit team technical specialists and QA auditors performed a limited 

review of various Study Plans and technical baseline documents including: 

(1) Waste Management System Description; 

(2) Waste Management System Requirements (WMSR); 

(3) Study Plans (SPs) for Calcite and Opaline Silica Vein Deposit 

Studies (SCP 8.3.1.5.2.1.5) and Midway Valley Faulting Studies 

(SCP 8.3.1.1.7.4.2); and 

(4) Technical Requirements Documents. 

The audit team developed separate checklists for the HQ and YMPO portions 

of the audit, based upon the OCRWM QARD and applicable procedures. The 

technical portion of the audit was generally focused on adequacy of 

documents and document review packages in establishing technical requirements 

and the traceability of such requirements, rather than the correctness 

(technical adequacy) of the requirements. 

HQ 

The auditors and technical specialists conducted joint interviews with 

the HQ personnel. During the initial portion of the audit, a considerable 

amount of time was spent with the audit team interviewing HQ personnel to 

come to an understanding of the activities being conducted. The portion of 

the audit observed consisted of discussion of technical document preparation 

Sprocedures. Use of the audit checklist was limited. Several of the 

checklist questions were revised to reflect recently issued procedures and 

to more accurately evaluate activities. 

ULL TEXTWASCII SCAN
\- /

9 

WMSR Volume IV, Revision 1 

The audit team reviewed records for the development, review and resolution 

of comments on WMSR Volume IV, and interviewed the HQ personnel associated 

with the document. The Technical Adequacy Assessment Group (TAAG) review 

records were also evaluated by the audit team. 

The documentation was sometimes inadequate, with unclear comment resolution 

and some illegible pages in document packages. The audit team was unable 

to consistently trace incorporation of review comments into the WMSR Volume IV. 

The HQ personnel responsible for the document had been assigned recently, 

and were not always able to adequately supplement the written record. 

The audit team was well prepared, knowledgeable about requirements for 

documentation, and persistent in both the document reviews and interviews. 

In a few instances, it appeared to the NRC observers that the auditors had 

been involved in work closely related to that which was being reviewed, 

but this possible lack of complete independence did not appear to adversely 

affect the audit process. 

The audit technical specialists encountered significant difficulties in 

attempting to understand how technical activities are conducted and 

controlled at OCRWM Headquarters. Therefore, the implementation of QA 

controls over the technical baseline activities should be considered 

indeterminate. 

YMPO 

Technical Requirements for Yucca Mountain Project (Midway Valley 

Trenching and Calcite/Silica Activities), YMP/CM-0007, Rev. 1 

The auditors and technical specialists performed a thorough, in-depth 

review of the Technical Requirements Document. The review was based on the 

technical checklist, but the audit team was aggressive in departing from 

the checklist to clear up questions. 

The focus of the review observed by the NRC staff was the traceability of 

technical requirements in the Technical Requirements Document. The audit 

team found the traceability of flow down of requirements through the 

heirarchy of documents to be poor, and documentation often inadequate. 

The YMPO technical personnel responsible for the Technical Requirements 

Document were not clear on the scope of their responsibilities, and there 

appeared to be items which may not have been assigned. The training 

received by YMPO personnel on procedures related to the Technical 

Requirements Document appeared weak, perhaps due in part to the recent 

issue of a number of these procedures (AP-3.3Q, Rev. 2, 10/17/90; 

QMP-03-09, Rev. 0, 10/17/90; QMP-06-04, Rev. 1, 10/17/90).

FULL TE*T ASCII SCAN
10

The NRC observers consider the control of development and review of the 

Technical Requirements Document by YMPO to be inadequate, based on the 

sample reviewed during the audit. 

(d) Instructions, Plans, Procedures, and Drawings (Criterion 5-HQ only) 

The audit of Instructions, Plans, Procedures, and Drawings consisted of 

interviewing HQ staff and reviewing Implementing Line Procedures (ILPs) 

5-1, 6-1 and 16-1. Several concerns were identified during the review of 

the stated ILPs. First, in reviewing ILP 6.1, it was found that changes 

made to the documents were not always recorded on the revision record. It 

was determined that only those changes considered to be "major" changes 

were recorded on the record. Second, in an effort to determine the definitions 

of "major" and "minor" changes, it was discovered that the definition of "minor change" as presented in ILP 5.1.1 is inconsistent with the definition 

presented in other procedures and in the supplement to NQA-1. 

The auditors conducted the interview and audit in a professional manner and 

probed beyond the checklist questions as appropriate. Although the auditors 

did not have the new QARD and QAPD sufficiently before the audit, they were 

familiar with the new procedures. In spite of the late release of the new 

QARD and QAPD, the audit is considered to be effective. However, due to the 

concerns identified above, the effectiveness of procedure implementation is 

indeterminate at this time. 

(e) Procurement Document Control (Criterion 4) and Control of Purchased 

Items and Services (Criterion 7) 

HQ 

The portion of the procurement control audit which was observed consisted 

of approximately equal amounts of interview with HQ personnel and review 

of documentation. The only procurement documents available to review were 

those for several services contracts, in which all personnel are operating 

under the OCRWM QA program. As such, QA program qualification is not 

applicable to this type of supplier. 

While procurement document controls could be evaluated, most of the controls 

of Criterion 7 were not applicable, so the effectiveness of its implementation 

is necessarily indeterminate. Although the auditors identified a deficiency 

in procurement pre-planning, it was reported during the audit team caucus 

that the implementation of procurement controls was effective. Considering 

the finding identified and the very small sample available for evaluation, 

the NRC observers consider that implementation of Criterion 4 is indeterminate 

and that the effectiveness conclusion stated by the auditors was premature. 

f FULL TEXT ASCII SCAN
11

YMPO 

Although the auditors utilized their checklists in reviewing Criteria 4 

and 7, there has been little or no work in these areas by YMPO. There 

have been no major procurements since 1987. The procurement action in 

1987 was for the award of the SAIC contract. Most of the items on the 

checklists were inconclusive as there was no implementation of activities 

pertaining to the recent revision of the QAPD and QMP's 04-01, 07-03, and 

07-04. 

To date, the YMPO QA organization has not qualified any supplier of items 

or services. This activity has been performed in the past by T&MSS of 

SAIC. After the DOE/YMPO audit of SAIC the week of November 11, 1990, the 

Qualified Suppliers List (QSL) of SAIC will be accepted by YMPO who will 

then be responsible for maintaining the QSL and performing future audits 

of suppliers. All major procurement activities in the future will be done 

by YMPO in accordance with QMP-04-02 which will incorporate the federal 

procurement practices (i.e., "Competition in Contracting Act" of 1984). 

Due to the absence of objective evidence available for review, this part 

of the audit was not effective in evaluating implementation. Therefore, 

before a conclusion can be reached on the adequacy of the YMPO controls 

and the competency and cognizance of YMPO staff in their programmatic 

responsibilities can be determined, additional audits or surveillances 

will have to be performed. 

(f) Identification and Control of Materials, Parts, Components, and Samples 

(Criterion 8); Control of Measuring and Test Equipment (Criterion 12); 

Handling, Storage, and Shipping (Criterion 13-YMPO only) 

During the initial portion of the audit, the auditors and technical 

specialists conducted interviews with key Sample Management Facility (SMF) 

personnel to come to a full understanding of the activities being conducted. 

The audit consisted of discussion of technical procedures, activities of 

the Sample Overview Committee, application of calibration controls, methods 

of controlling SMF quality-affecting activities, and a "walkthrough" 

explanation of the work areas at the SMF. The review of objective evidence 

was limited because previous surveillance activities had evaluated SMF 

controls. During the audit, the prepared checklist was used as the guide for auditor questions and discussions. Overall, the auditors and technical 

specialists were thorough in their evaluation. 

The observers, who were less familiar with the SMF activities than were the 

auditors, were taken on a short facility tour to learn of the physical 

controls designed to ensure traceability of core samples. The SMF personnel 

displayed a knowledge of the procedures and intent of the QA controls, and 

they demonstrated effective implementation of the controls.

FULL TEXT ASCII SCAN
12

(g) Corrective Action (Criterion 16) 

The checklist used for the corrective action review was adequate and the 

auditors conducted a thorough, lengthy review and detailed probe of the 

activities. The auditors were well prepared and pursued beyond the 

checklist with substantive questions. 

Deficiency Reports (DRs) and Corrective Action Reports were reviewed for 

determination of root cause and timeliness of closeout. The auditors noted 

instances where corrective action was not always accomplished in a timely 

manner. Certain of the discrepancies had been in the tracking system in 

excess of a year with little or no actions to close them. In addition, 

instances were noted where some of the reports contained an inadequate 

description of the root cause of the deficiency, and others contained no 

description at all of the root cause. 

The procedure governing Corrective Action Requests (CARs), QAAP 16.1, 

Revision 2, was effective October 15, 1990. It was not clear to the NRC 

observers that the HQ personnel were familiar with the procedure require

ments, and there was no consideration given to resolving DRs still open 

under the previous procedure. 

The auditors also reviewed and evaluated DOE's trending procedures. It 

was determined through interviews with HQ staff that HQ did not perform 

any trending analysis in 1990 as required by QAAP 2.9. Further, HQ 

management acknowledged that they were knowingly in non-compliance with 

the procedure requirements. 

Based on the information obtained during the audit it must be concluded 

that the audit process was effective, but the implementation of procedures 

by HQ is ineffective. 

YMPO 

The corrective action portion of the audit which was observed consisted of 

a sampling of documentation in the form of completed Standard Deficiency 

Reports (SDRs). Although a few random SDRs were selected, the majority of 

the SDRs were suggested by audit team members or they were selected based 

on information the auditors had gained from earlier audits. This was an 

appropriate approach which focused the audit on the most important SDRs. 

The NRC observers reviewed about one-half of the audited SDRs and concurred 

with the auditors' conclusion that implementation was generally effective. 

FULL TEXT ASCO SCAN
13

Based upon the auditors' review of the CA criterion, this portion of the 

audit was conducted in an effective manner. The personnel involved with 

maintenance and processing of the CA records seemed to be experienced, 

capable, and familiar with their QA responsibilities. 

(h) Quality Assurance Records (Criterion 17) 

HQ 

The Central Records Facility (CRF), which is run by a contractor (Koh 

Systems, Inc.) for DOE, was not part of the scope of this audit. The 

Quality Records Center (QRC) at HQ was opened in June 1990. It receives 

document packages from OCRWM personnel, reviews them for completeness, 

obtains missing documents required to complete the packages, and forwards 

completed packages to the CRF for permanent storage. About ten record 

packages had been processed and accepted as complete by the QRC at 

the time of the audit. The QRC records storage facility consisted of two 

metal file cabinets which were not fire rated, did not meet NQA-1 

requirements, and were thus inadequate. 

The auditors were obviously familiar with the procedural requirements 

applicable to records receipt and storage (QAAP 17.1 and IP 12.17.01), 

and had prepared a detailed checklist. The audit of this area was 

thorough, and the auditors were persistent in following up on questions. 

Implementation of QRC precedures was too limited to determine its 

effectiveness. 

YMPO 

The YMPO Local Records Center personnel involved with the observed portion 

of the Criterion 17 audit appeared familiar with applicable procedures and 

QA requirements and were able to retrieve requested records promptly. 

Based upon the document retrieval observed and explanations provided, the 

QA Records procedures appear to be adequately implemented. 

The auditors effectively utilized their checklists, evaluated a significant 

sample, and were extremely thorough in their investigations. 

(I) Audits (Criterion 18) 

HQ 

The audit of this criterion consisted of interviewing HQ staff and evaluat

ing several DRs. The reports reviewed included DR-90-008 and DR-90-014. 

The auditors identified a number of deficiences with regard to Criterion 18. 

They included: 

1. HQ did not conduct any internal audits or surveillances In 1990. 

This means that no action was taken to correct the deficiencies 

described in DR-90-014. 

NS ONI
14

2. The recommended action for DR-90-014 was to schedule and perform 

audits in accordance with QAAP 18.2. There is no objective evidence 

to indicate that a final audit schedule was ever prepared. 

3. QAAP 18.3 requires that checklists be prepared from the requirements 

being surveilled. Evaluations of five 1989 surveillance reports 

indicate that the "Requirements Surveilled" section of the reports 

reference procedures which are not referenced in the checklists. 

The auditors did an effective job of evaluating the OCRWM HQ QA program in 

the areas of corrective action and audits. The prepared checklist was 

used and when appropriate the auditors probed with in-depth questions. 

Implementation by OCRWM HQ under this criterion appears to be ineffective. 

YMPO 

The NRC observers were involved only in the audit portion of Criterion 18 

pertaining to surveillances. The auditors reviewed the surveillance 

schedules for FY-90 and FY-91 and verified that the schedules were maintained 

as required. A review of a number of surveillance packages indicated that 

the surveillance process complied with procedure QMP-18-02 with regards to 

the use of checklists, documentation of results, and generation of SDRs, 

Non-Conformance Reports, and/or observations as applicable. The auditor 

verified that those individuals who participated as surveillance team members 

had the training and experience to be qualified in accordance with procedure 

QMP-02-02. 

The YMPO program for surveillances, as represented by the sample observed 

during this audit, appeared to be well planned, implemented, and generally 

effective. The audit of this area was thorough and professional In nature, 

emphasizing the use of objective evidence to support statements made by 

YMPO QA personnel. 

5.4 Conduct of Audit 

The QA and technical portions of the audit were productive and performed 

in a professional manner. Despite the late release of the QARD and QAPD, 

the audit team was generally prepared and demonstrated a sound knowledge 

of the QA aspects of the program. The audit checklists included the 

important QA controls addressed in the QARD. The audit team used the 

comprehensive checklists effectively during the interviews with personnel 

and review of documents. In general, the team was persistent in their 

interviews, challenging responses when necessary. The integration of the 

technical and programmatic portions of the audit was effective. 

5.5 Qualification of Auditors 

The qualifications of the QA auditors on the team were previously accepted 

by the NRC staff (ref. NRC Observation Audit Report for USGS dated 

August 22, 1988) or were acceptable based on their meeting the requirements 

of QMP-02-02, the YMPO procedure for qualifying auditors. 

FULL TET ASII SCAN
15

5.6 Audit Team Preparation 

In general, the QA auditors and technical specialists were well prepared in 

the areas they were assigned to audit and knowledgeable in the QARD and 

implementing procedures. Overall, Audit Plan 90-1-01 was complete and 

included: (1) the audit scope; (2) a list of audit team personnel; (3) a 

list of the audit activities; (4) the audit notification letter; (5) the 

QARD and QAPD; and (6) the QA and technical checklists. 

5.7 Audit Team Independence 

The audit team members did not have prior responsibility for performing 

the activities they investigated. As discussed in Section 5.3(c), there 

was some question of total independence of some auditors; however, members 

of the team appeared to have sufficient independence to carry out their 

assigned functions in a correct manner without adverse pressure or 

influence. Since this was an internal audit, the NRC staff believes 

sufficient independence of audit team members was demonstrated. 

5.8 Summary of NRC Staff Findings 

(a) Observations 

The NRC staff did not identify any observations relating to deficiences 

in either the audit process or the other elements of OCRWM QA program 

implementation. 

(b) Weaknesses 

Some auditors appeared to spend a disproportionate time conducting 

interviews rather than evaluating objective evidence, especially 

during the HQ portion of the audit (Refer to Section 5.3(a)). 

The NRC staff believes that the timing of the audit was less than 

optimal. In some cases, audit checklists were revised up to and after 

the start of the audit to incorporate requirements from procedures 

issued just prior to the start of the audit. Further, due to the 

recent reorganization within HQ, the auditors in several instances were 

obligated to interview both the personnel currently assigned and those 

formerly assigned to various functions (Refer to Sections 5.2 and 

5.3(c)). 

Several of the auditors were OCRWM HQ (or HQ contractor) personnel, 

and on more than one occasion they appeared more knowledgeable of the 

activity being audited than the individual being interviewed. These 

auditors may have been of greater value as auditees. Otherwise, OCRWM 

and contractor personnel appeared to be competent and generally familiar 

with QA requirements and their respective responsibilities (Refer to 

Section 5.3(c)). 

FULL TEXT ABCA SCAN

11 II
16

A preliminary effectiveness conclusion concerning Criteria 4 and 7 

presented by an auditor during a status meeting did not appear to be 

well supported by the available objective evidence (Refer to 

Section 5.3(e)). 

No annual management assessment of the HQ QA program was performed 

(Refer to Section 5.3(a)). This is similar to findings from previous 

audits of High Level Waste (HLW) repository program participants. 

The HQ QA Division was not completely staffed prior to and at the 

time of the audit (Refer to Section 5.3(a)). 

There were indications that training was inadequate in some areas 

(Refer To Sections 5.3(b) and (c)). 

No trending analyses had been performed (Refer to Section 5.3(g)). 

This is similar to findings from previous audits of HLW repository 

participants. 

The HQ CA program did not result in timely and effective closure of 

conditions adverse to quality (Refer to Section 5.3(g)). This is 

similar to findings from previous audits of HLW repository 

participants. 

There appeared to be inadequate review of DRs and CARs for root 

cause and generic implications (Refer to Section 5.3(g)). This is 

similar to findings from previous audits of the HLW repository 

program. 

The HQ program for internal audits/surveillances was inadequate and 

ineffective (Refer to Section 5.3(i)). This is similar to findings 

from previous audits of the HLW repository program. 

The problems identified by the audit team with the WMSR Vol. IV and 

the YMP/CM-0007, Rev. 1 indicates additional management attention 

is needed in these technical activities (Refer to Section 5.3(c)). 

Based on the above, the NRC observers determined that OCRWM management 

had not adequately evaluated the results of prior audits of the HLW 

repository program and applied the lessons learned from these audits 

to the OCRWM QA program. 

(c) Good Practices 

In general, the auditors and technical specialists used well 

researched and detailed checklists and extended their investigations 

beyond the checklists when appropriate. Integration of programmatic 

and technical portions of the audit was effective due to the 

simultaneous conduction of the programmatic audits of Criteria 3 and 

20 with the technical evaluations. 

F.tL T.XT AsCII SCAN

it 11
17

Daily caucuses were held between auditors and observers, and daily 

meetings were held between OCRWM management and the Audit Team Leader 

to discuss potential findings. Auditors identifying potential findings 

were included in these status meetings to more clearly explain deficient 

conditions and allow for resolution during the audit as much as possible. 

Findings were well substantiated and reflected significant rather than 

trivial Issues. The audit team also did a good job of answering observer 

questions as they were raised. 

5.9 Summary - DOE/YMPO Audit Team Findings 

At the formal exit briefing on October 31, 1990, the audit team identified 

18 potential CARs written against the OCRWM QA program (11 to HQ and 7 to 

YMPO). In addition, during the audit, OCRWM was able to resolve 29 remedial 

deficiencies (11 at HQ and 18 at YMPO). The CARs issued to OCRWM can be 

summarized as follows: 

(a) A matrix to cross reference OCRWM procedures & QAPD to QARD does 

not exist (YMPO). 

(b) There are inadequate controls to assure training of personnel prior 

to initiation of quality affecting activities (YMPO). 

(c) The flow down of WMSR Volume IV requirements to other requirements 

documents is not apparent (YMPO). 

(d) Inputs to Technical Requirements Document YMP/CM-O007 are not always 

traceable (YMPO). 

(e) The review process for YMP/CM-0007 is deficient (YMPO). 

(f) There was a lack of control procedures for development of 

YMP/CM-0007 since QMP-03-09 was implemented following completion 

and processing (YMPO). 

(g) Interim Change Notices were incorrectly classified as "minor" changes 

(YMPO). 

(h) A draft of QAAP 2.2 was issued for use prior to formal review and 

approval (HQ). 

(I) Approval of potential interfaces was not in conformance with 

QAAP 3.7, Rev. 1 (HQ). 

(j) TAAG comment sheets were not signed by the TAAG chairperson (HQ). 

(k) There was a lack of procedure for addressing review comments for 

dependent volumes of the WMSR (HQ). 

RILL TEXT ASCfl SCAN

II 11
18

(1) There is inadequate definition of "minor change" in QAAP 5.1, 

Rev. 2 and QAAP 5.2, Rev. 1 (HQ). 

(m) Revisions made to QAAP's 6.1 and 16.1 were not traceable in the 

.revision record (HQ). 

(n) Control requirements for the WMSR and WMSD technical management plans 

are not consistent with stated requirements (HQ). 

(o) The reports for tracking deficiencies and monthly actions are 

ineffective (HQ). 

(p) No description of the QRC is present in ILP-12-17.01, and the current 

storage facility does not meet minimum requirements (HQ). 

(q) Ineffective implementation of procedural requirements for auditors 

and technical specialists (HQ). 

(r) Ineffective implementation of verification activities (HQ). 

The audit team also found implementation of all, but three of the Criteria 

reviewed to be ineffective or indeterminate at HQ, or YMPO, or both. 

Along with the deficiencies identified above, the audit team also recommended 

at the formal exit briefing that the OCRWM Office of Quality Assurance conduct 

several surveillances to verify the effectiveness of the QA program prior to 

the start of any new site characterization activities. The recommended areas 

to be surveyed include: 

(a) control of technical baseline (HQ), 

(b) corrective action system (HQ), 

(c) Quality Records Center (HQ), 

(d) program overview (HQ), 

(e) preparation and review of technical requirements for YMP (YMPO), 

(f) SNL activities related to YMP/CM-0007 (YMPO), and 

(g) training (YMPO). 

The NRC staff fully concurs with these recommendations. 

UTEXT ASCII SCAN

11 11