March 28, 2000

Mr. Dwight Shelor, Acting Director

Program Management and Administration

Office of Civilian Radioactive Waste Management

U.S. Department of Energy

1000 Independence Avenue, SW

Washington, DC 20585

SUBJECT: U.S. NUCLEAR REGULATORY COMMISSION’S OBSERVATION AUDIT

REPORT NO. OAR- 00-06, “OBSERVATION AUDIT OF OFFICE OF THE

CIVILIAN RADIOACTIVE WASTE MANAGEMENT, QUALITY ASSURANCE

DIVISION, AUDIT NO. M&O-ARP-00-006"

Dear Mr. Shelor:

I am transmitting the U.S. Nuclear Regulatory Commission’s (NRC’s) Observation Audit Report

No.OAR-00-06 of the U.S. Department of Energy (DOE), Office of Civilian Radioactive Waste

Management (OCRWM), Office of Quality Assurance (OQA), Yucca Mountain Quality

Assurance Division, audit of the Engineered Barrier System (EBS) Process Model Report

(PMR) activities performed by the OCRWM Management and Operating Contractor (M&O).

The audit, M&O-ARP-00-006, was conducted on February 7–11, 2000, at the M&O facilities in

Las Vegas, Nevada.

The scope of the audit was limited to evaluating the effectiveness of the implementation of the

OCRWM QA Program described in the Quality Assurance Requirements and Description and

its implementing procedures for selected analysis model reports (AMRs) supporting the EBS

PMR.

The NRC staff determined that this audit was effective in identifying deficiencies and

recommending improvements in the AMR process. During the conduct of the audit, both the

audit team and the NRC observers reviewed data, analysis and model reports, and software

within the scope of the audit to determine whether they were properly qualified. The audit team

determined that the AMRs were adequate considering their stage of development, but could be

substantially improved through incorporation of the recommendations provided by the audit

team. The NRC staff agrees with the audit team conclusions, findings, and recommendations.
D. Shelor -2-

A written response to this letter and the enclosed report is not required. However, we do

expect OQA to provide replies to the open Audit Observer Inquiries. If you have any questions,

please contact Bob Latta of my staff at (301) 415-5228.

Sincerely,

/RA/

C. William Reamer, Chief

High-Level Waste and Performance

Assessment Branch

Division of Waste Management

Office of Nuclear Material Safety

and Safeguards

Enclosure: NRC Observation Audit Report No. OAR-00-06, “Observation Audit of the Office

of Civilian Radioactive Waste Management, Quality Assurance Division, Audit

No. M&O-ARP-00-006"

cc See attached list
Letter to D. Shelor from C.W. Reamer dated: March 28, 2000

cc: R. Loux, State of Nevada

S. Frishman, State of Nevada

L. Barrett, DOE/Wash, DC

A. Brownstein, DOE/Wash, DC

S. Hanauer, DOE/Wash, DC

C. Einberg, DOE/Wash, DC

N. Slater, DOE/Wash, DC

R. Dyer, YMPO

S. Brocoum, YMPO

R. Clark, YMPO

C. Hanlon, YMPO

T. Gunter, YMPO

G. Dials, M&O

J. Bailey, M&O

D. Wilkins, M&O

M. Voegele, M&O

S. Echols, Winston & Strawn

B. Price, Nevada Legislative Committee

J. Meder, Nevada Legislative Counsel Bureau

D. Bechtel, Clark County, NV

E. von Tiesenhousen, Clark County, NV

J. Regan, Churchill County, NV

H. Ealey, Esmeralda County, NV

L. Fiorenzi, Eureka County, NV

A. Remus, Inyo County, CA

T. Manzini, Lander County, NV

J. Pitts, Lincoln County, NV

J. Wallis, Mineral County, NV

L. Bradshaw, Nye County, NV

M. Murphy, Nye County, NV

J. McKnight, Nye County, NV

N. Stellavato, Nye County, NV

K. Kirkeby, White Pine County, NV

D. Weigel, GAO

W. Barnard, NWTRB

R. Holden, NCAI

C. Williams, NIEC

R. Arnold, Pahrump County, NV

J. Lyznicky, AMA

R. Clark, EPA

F. Marcinowski, EPA

R. Anderson, NEI

R. McCullum, NEI

S. Kraft, NEI

J. Kessler, EPRI

R. Wallace, USGS

R. Craig, USGS

W. Booth, Engineering Svcs, LTD

J. Curtiss, Winston & Strawn
D. Shelor -2-

A written response to this letter and the enclosed report is not required. However, we do expect

OQA to provide replies to the open Audit Observer Inquiries. If you have any questions, please

contact Bob Latta of my staff at (301) 415-5228.

Sincerely,

/RA/

C. William Reamer, Chief

High-Level Waste and Performance

Assessment Branch

Division of Waste Management

Office of Nuclear Material Safety

and Safeguards

Enclosure: NRC Observation Audit Report No. OAR-00-06, “Observation Audit of the Office

of Civilian Radioactive Waste Management, Quality Assurance Division, Audit

No. M&O-ARP-00-006"

cc: See attached list

DISTRIBUTION:

File Center PUBLIC ACNW LSS CNWRA NMSS r/f HLWB r/f

SWastler CGlenn BMabrito WBelke JHolonich Khooks HArlt

DBrooks JHolonich JGreeves RCodell RLatta TBloomer

DOCUMENT NAME: S/DWM/HLWB/thc1/OAR-00-06.EBS

*SEE PREVIOUS CONCURRENCE

OFC HLWB* HLWB* HLWB HLWB

NAME TCarter:vlm LCampbell NKStablein CWReamer

DATE 03/23/00 03/23/00 03/28/00 03/28/00

OFFICIAL RECORD COPY

ACNW: YES NO Delete file after distribution: Yes No

1) This document should/should not be made available to the PUBLIC _____ ______

(Initials) (Date)

2) This document is related to the HLW program. It should be placed in the LSS. _____ ______

(Initials) (Date)
1

U.S. NUCLEAR REGULATORY COMMISSION

OBSERVATION AUDIT REPORT NO. OAR-00-06

OBSERVATION AUDIT OF THE

OFFICE OF CIVILIAN RADIOACTIVE WASTE MANAGEMENT

QUALITY ASSURANCE DIVISION

AUDIT NO. M&O-ARP-00-006

/S/ 03/22 /00 /S/ 03/28/00

Richard Codell Robert Latta

High-Level Waste and Performance High-Level Waste and Performance

Assessment Branch Assessment Branch

Division of Waste Management Division of Waste Management

/S/ 03/22/00 /S/ 03/15/00

Tamara Bloomer Hans Arlt

High-Level Waste and Performance High-Level Waste and Performance

Assessment Branch Assessment Branch

Division of Waste Management Division of Waste Management

/S/ 03/15/00 /S/ 03/15/00

Robert Brient Goodluck Ofoegbu

Center for Nuclear Waste Regulatory Center for Nuclear Waste Regulatory

Analyses Analyses

Reviewed and Approved by:

/S/ 03/28 /00

N. King Stablein, Chief

Projects and Engineering Section

High-Level Waste and Performance

Assessment Branch

Division of Waste Management

1.0 INTRODUCTION
2

Staff from the U.S. Nuclear Regulatory Commission (NRC) Division of Waste Management and from

the Center for Nuclear Waste Regulatory Analyses (CNWRA) observed the U.S. Department of

Energy (DOE), Office of Civilian Radioactive Waste Management (OCRWM), Office of Quality

Assurance (OQA), Yucca Mountain Quality Assurance Division performance-based audit of the

Engineered Barrier System (EBS) Process Model Report (PMR) activities performed by the OCRWM

Management & Operating Contractor (M&O). The audit, M&O-ARP-00-006, was conducted on

February 7–11, 2000, at the M&O facilities in Las Vegas, Nevada.

The objective of this audit was to evaluate the implementation of the applicable provisions contained in

the OCRWM Quality Assurance Requirements and Description (QARD), DOE/RW-0333P, Revision 8,

by reviewing selected Analysis Model Reports (AMRs) supporting the EBS PMR. During the audit,

selected AMRs were subjected to a technical and programmatic review to ensure that the applicable

requirements contained in the QARD were met.

The NRC staff objective was to gain confidence that the M&O and OQA are properly implementing the

provisions contained in the QARD and the requirements contained in Subpart G, Quality Assurance, to

Part 60, of Title 10 of the Code of Federal Regulations (10 CFR Part 60). Because of the anticipated

DOE submittal of the site recommendation (SR) in November 2000, the following observation activities

were emphasized: (1) confirming that data, software, and models supporting SR are properly qualified;

and (2) reviewing the progress being made by DOE and its contractors in meeting the qualification

goals for SR.

This report addresses the NRC staff determination of the effectiveness of the OQA audit and the

adequacy of implementation of QARD controls by the M&O in the audited areas of AMR development.

2.0 MANAGEMENT SUMMARY

The NRC staff has determined that OQA Audit M&O-ARP-00-006 was useful, effective, and conducted

in a professional manner. Audit team members were independent of the activities they audited and

appeared to be knowledgeable in the QA and technical disciplines within the scope of the audit. The

audit team members’ qualifications were reviewed and the members were found to be qualified in their

respective disciplines.

The audit team concluded the OCRWM QA program had been satisfactorily implemented in the areas

evaluated. However, six apparent deficiencies were identified during the audit, and approximately

eighteen recommendations were offered for improvements and enhancements to the AMRs.

The NRC staff determined that this audit was effective in identifying deficiencies and recommending

improvements in the AMRs. During the conduct of the audit, both the audit team and the NRC

observers reviewed data, analysis reports, and software within the scope of the audit to determine

whether they were properly qualified. The audit team and the NRC observers determined that certain

software supporting the AMRs, had not been properly qualified. The team also noted that most of the

data were categorized as “to be verified.” The NRC staff agrees with the audit team conclusions,

findings, and recommendations. The AMRs were adequate for their current early stage of

development, but could be substantially improved in clarity and justification of assumptions and

technical positions taken with incorporation of the audit team recommendations.

NOTE: [Subsequent to the audit, OQA decided to postpone the remaining 3 PMR audits that

were scheduled to be performed during the months of March and April 2000. This
3

decision was made, in part, because several AMR completion dates had slipped and

lessons learned from the previous 6 PMR audits needed to be communicated to the

preparers of the AMRs.]

3.0 AUDIT PARTICIPANTS

3.1 Nuclear Regulatory Commission Observers

Robert Brient Observer (Team Leader-CNWRA)

Richard Codell Observer (Technical Specialist-NRC)

Robert Latta Observer (Senior QA Engineer-NRC)

Tamara Bloomer Observer (Technical Specialist-NRC)

Hans Arlt Observer (Technical Specialist-NRC)

Goodluck Ofoegbu Observer (Technical Specialist-CNWRA)

3.2 Office of Quality Assurance Audit Team

Donald Harris Audit Team Leader OQA/Quality Assurance Technical Support

Services (OQA/QATSS)

Richard Weeks Auditor OQA/QATSS

Stephen Harris Auditor OQA/QATSS

Emily Jensen Auditor OQA/QATSS

George Harper Auditor OQA/QATSS

Harris Greenberg Technical Specialist DOE/Management and Technical Services

(MTS)

David Sassani Technical Specialist DOE/MTS

Steve Sobkowski Technical Specialist DOE/MTS

Arthur Stein Technical Specialist DOE/MTS

4.0 REVIEW OF THE AUDIT AND AUDITED ORGANIZATION

This OQA audit of the M&O was conducted in accordance with OCRWM Quality Assurance Procedure

(QAP) 18.2, “Internal Audit Program,” and QAP 16.1Q, “Performance/Deficiency Reporting.” The NRC

staff’s observation of this audit was based on the NRC draft procedure, “Conduct of Observation

Audits,” issued October 6, 1989 (Draft).

4.1 Scope of the Audit

The audit team conducted a limited scope, performance based audit of activities and processes related

to the development of the AMRs supporting the EBS PMR. AMRs, software, and data were evaluated

during the audit process. The audit included review of the programmatic controls governing the AMRs

and technical issues discussed in the AMRs. Specifically, the following procedures and AMRs

supporting the EBS PMR were reviewed by the audit team and the NRC observers during the audit:

Procedures

a) AP-2.1Q, “Indoctrination and Training Personnel,” Revision 0, with Interim Change Notice (ICN)

No. 0

b) AP-2.13Q, “Technical Product Development Planning,” Revision 0, with ICN No. 1
4

c) AP-SI.1Q, “Software Management,” Revision 2, with ICN No. 0

d) AP-3.15Q, “Managing Technical Product Inputs,” Revision 0, with ICN No. 1

e) AP-SIII.2Q, “Qualification of Unqualified Data and the Documentation of Rationale for Accepted

Data,” Revision 0, with ICN No. 0

f) AP-3.10Q, “Analysis and Models,” Revision 1, with ICN No. 0

g) AP-2.14Q, “Review of Technical Products,” Revision 0, with ICN No. 0

h) AP-SIII.3Q, “Submittal and Incorporation of Data to the TDMS,” Revision 0

i) YAP-SV.1Q, “Control of the Electronic Management of Data,” Revision 0, with ICN No. 1

j) QAP-SIII-1, “Scientific Investigations,” Revision 3

Analysis Model Reports

a) ANL-EBS-MD-000020, “In-Drift Corrosion Products,” Revision 00

b) ANL-EBS-MD-000026, “In-Drift Thermal-Hydrological-Chemical Model,” Revision 00

c) ANL-EBS-MD-000075, “Ventilation Model,” Revision 00

d) ANL-EBS-MD-000080, “Drift Degradation Analysis,” Revision 00

4.2 Conduct and Timing of the Audit

The audit was performed in a professional manner and the audit team demonstrated a sound

knowledge of the applicable M&O and DOE programs and procedures. Audit team personnel were

persistent in their interviews, challenged responses when appropriate, and performed an acceptable

audit. Due to time constraints during the preparation phase, the programmatic and technical portions

of the audit were not conducted simultaneously as has been common practice. While this did not

adversely impact the audit overall, better integration of the programmatic and technical elements may

have helped the technical specialists, who appeared to be relatively inexperienced in expressing their

findings.

The DOE audit team and NRC observers caucused at the end of each day. Also, meetings of the audit

team and M&O management (with the NRC observers present) were held each morning to discuss the

current audit status and preliminary findings.

The NRC staff believes the timing of the audit was appropriate for the auditors to evaluate ongoing

EBS AMR/PMR activities, however; recurring findings of this and other AMR/PMR audits suggest that

improvements to the AMR/PMR development process may be appropriate before additional audits are

conducted.
5

4.3 Audit Team Qualification and Independence

The qualifications of the DOE audit team leader and the OQA audit team members were found to be

acceptable in that they met the requirements of QAP 18.1, “Auditor Qualification,” as verified by the

NRC observation audit lead. The audit team members did not have prior responsibility for performing

the activities they audited. In addition, training, education and experience records for audit team

members were reviewed and found acceptable. The NRC observers noted that this was the first

experience for the audit team technical specialists and to some degree, they had difficulty in

expressing their AMR technical comments and recommendations as findings on the AMR

development/review processes and/or on the technical quality of the AMRs. By the completion of the

audit, most of the technical specialists had resolved these difficulties.

4.4 Examination of Quality Assurance Elements

The OQA programmatic and technical audit activities were conducted separately. The limited scope

audit focused on the QA elements closely associated with the development of the AMRs. The NRC

observation team evaluated the audit team’s review of the following QA elements.

4.4.1 AP-2.13Q “Technical Product Development Planning”

The DOE auditors reviewed technical development plans (TDPs) and work product planning sheets

(WPPS) applicable to the subject AMRs. A deficiency was identified in the planning documents for the

In-Drift Thermal-Hydrological-Chemical Model that had not been revised to reflect the current situation

and to reflect the true scope of the AMR.

4.4.2 AP-SI.1Q “Software Management”

Some of the software used in support of the AMRs was not qualified or controlled in accordance with

procedure AP-S1.1Q. The Ventilation Model and Drift Degradation calculations performed using

spreadsheet (EXCEL and Mathcad) software were not documented and controlled as required. The

DRKBA version 3.3 software, also used in the Drift Degradation Analysis, was not qualified, and the

required Software Activity Plan was not prepared. Two processors for the NUFT code used in the InDrift Thermal-Hydrological-Chemical Model, RADPRO and XTOOL, had not been qualified, and the

output of the analysis had not been classified as to-be-verified (TBV) as required. In addition, one

module of the (otherwise qualified) NUFT software used in this model was not qualified.

4.4.3 AP-3.15Q “Managing Technical Product Inputs”

Each of the AMRs examined included document input reference sheets that list the inputs to and

references cited in the AMR. The document input reference sheets also identified the status of the

input (e.g., qualified, TBV).

The status of the input documents for the AMRs is summarized as follows:

• Most of the data used are identified as TBV, with the reason as “unconfirmed.” These are data

that had been collected under the OCRWM QA program, but were placed in the TBV status due

to a corrective action request that resulted in the data qualification being of indeterminate

quality. The M&O has plans for confirming these data and removing the TBV; however, that

has not been accomplished for the data affecting these AMRs.

• A few data in the AMRs are classified as TBV and unqualified. These data were not collected

under the QA program, and require formal qualification.
6

• Some data are classified as N/A, used for reference only.

4.4.4 AP-SIII.2Q “Qualification of Unqualified Data and the Documentation of Rationale for

Accepted Data”

As determined during the audit/observation the qualification process had not been initiated for the

unqualified data used in these AMRs.

4.4.5 AP-3.10Q “Analysis and Models”

The four AMRs evaluated during this audit are classified as follows:

• In-Drift Corrosion Products - Conceptual Model

• In-Drift Thermal-Hydrological-Chemical Model - Model Documentation

• Ventilation Model - Conceptual Model

• Drift Degradation Analysis - Analysis

All four of these reports had been issued as Revision 00 and the development and technical checking

processes described in AP-3.10Q had been completed.

The AMRs had been subjected to the technical checking process. Two potential deficiencies were

identified associated with this process:

• Some technical checker comments on the In-Drift Thermal-Hydrological-Chemical Model AMR

were not resolved and were deferred to a later revision. The procedure has no provision for

deferring mandatory comments.

• The qualification documentation for the checker and a technical reviewer of the In-Drift

Corrosion Products AMR did not reflect technical competencies in the AMR subject matter.

4.4.6 AP-2.14Q “Review of Technical Products”

The AMRs reviewed were subjected to the technical review process. In particular, the AP-2.14Q

technical reviews are performed by organizations outside of the author’s, so they may serve primarily

as interface reviews.

As indicated in paragraph 4.4.5, a potential deficiency was identified in the qualification documentation

for a technical reviewer of the In-Drift Corrosion Products AMR that did not reflect technical

competencies in the AMR subject matter.

4.4.7 YAP-SV.1Q “Control of the Electronic Management of Data”
7

The audit team identified a potential deficiency in the implementation of this procedure in the Process

Control Evaluations for the AMR activities. The evaluations failed to identify procedures that needed to

be revised and/or the identified procedures had not been revised.

4.4.8 AP-2.1Q “Indoctrination and Training of Personnel”

The audit team evaluated personnel qualification records for key individuals performing the AMR

development and review activities. Within this area, a potential deficiency was identified regarding

employees that had transferred from other departments/labs that had been “grand- fathered” for

previous training, but had not had their training re-baselined by their new managers.

4.5 Examination of Technical Activities

The DOE audit team prepared detailed checklists for each of the AMRs. Technical activities examined

by the audit team, and in some cases those questions forwarded to the audit team, are summarized

below for each of the AMRs. The DOE audit team and NRC observers identified a number of

weaknesses in the AMRs that could adversely affect their value in supporting licensing decisions.

However, the M&O intends for the “final” AMRs/PMRs to be fully justified and substantiated, with the

goal of minimizing NRC requests for additional information. The comments provided in the following

sections include the weaknesses. The AMRs, while adequate for their early stage of development,

would not meet the standards needed to support licensing.

4.5.1 Analysis Model Report In-Drift Corrosion Products (ANL-EBS-MD-000020 Rev 00)

The In-Drift Corrosion Products AMR described the conceptual model exploring the possible effects of

EBS corrosion products on the near-field environment and the geochemical environment. It may also

provide input into the unsaturated and saturated zone modeling efforts. As a conceptual model, the

AMR in Revision 00 was adequate, but future versions will need a greater level of detail and

justification for positions taken.

The audit of the In-Drift Corrosion Products AMR included procedural and technical inquiries to verify

that procedures were followed and that the quality of the product was satisfactory. The auditor inquired

about the technical basis for the report including: a) assumptions used, b) justifications for inclusions

and exclusions of elements within the conceptual model framework, c) conclusions, d) TDP, and e)

technical checker comments. The DOE auditor followed a prepared checklist and identified several

issues similar to those identified in the container life and source term (CLST) team review of the

document. While these issues would be important to the final revision of the AMR, the audit team

determined that the AMR was sufficient as a conceptual model.

The DOE audit team also evaluated the M&O technical checking and technical review processes. In

reviewing the technical checker comments, it appeared that the vast majority of comments were

editorial rather than technical in nature, which led to concerns regarding the checker’s credentials.

Furthermore, the documentation of the checker’s qualification did not appear to support his selection to

perform this task.

4.5.2 Analysis Model Report In-Drift Thermal-Hydrological-Chemical Model (ANL-EBS-MD000026, Rev 00)

The THC model considered only thermal and hydrological components. One of the stated goals of the

modeling was to estimate dripping within the drift. Because of time limitations, the investigators chose
8

to use a qualified model (NUFT), which employed an Equivalent Continuum Model (ECM) approach to

flow in the fracture/matrix system. A Dual Permeability Model (DKM) version of NUFT exists, but was

not a qualified code. The analyses should be updated for the PMR stage, although they do not expect

to make a new revision to the current AMR.

Convective heat transfer through the air space surrounding the waste package was added by

specifying an artificial hydraulic conductivity adjusted to fit a textbook solution for convective heat

transfer in coaxial heated cylinders. Actual dripping of water could not be modeled explicitly because

the model resolution was too coarse, and the ECM practically precludes fracture flow. Ultimately, the

main emphasis was not dripping from the drift onto the waste packages as implied, but rather

conditions that could lead to condensation of water and subsequent dripping under the drip shield.

Despite the limitations, the model could determine with reasonable reliability that the drip shield would

always remain hotter than the floor under the waste package (the invert). Under these conditions,

condensation under the drip shield is not likely.

Several shortcomings of the study were identified, including failure to follow the stated goals in the

planning documents, inconsistent representation of the system (e.g., the WP sits directly on the invert

instead of a pedestal), and not covering a wide enough range of possible conditions of heat loading,

backfill and drip-shield placement.

4.5.3 Analysis Model Report Ventilation Model (ANL-EBS-MD-000075, Rev 00)

The purpose of the ventilation model was to predict the fraction of heat that would be removed from the

repository during the preclosure stage. The analyses used a combination of two-dimensional models

for heat transfer in drift-normal planes, and spread-sheet calculation for along-drift heat transfer. The

numerical stability of the explicit stepping algorithm applied in the analyses to advance the solution

along the drifts was not investigated, which raises the possibility that the calculated air and drift-wall

temperatures, and, consequently the predicted amount of heat removal by ventilation, may not be

correct. This possibility was strengthened by the results of calculations performed by CNWRA staff to

check the consistency of the air and drift-wall temperatures given in the report. The two sets of

temperatures are inconsistent: the drift-wall temperatures were not reproduced by analyses that used

the air temperatures as input.

Because heat removal by ventilation is an important component of the thermal-load management

strategy that is currently being proposed by DOE, the analyses of the ventilation design should be

based on a rigorous and tested model. The model used in the report does not meet such a standard.

The authors of the AMR chose to investigate the ventilation process using qualified computer codes.

The AMR authors believed there was insufficient time to qualify other software. The outcome of this

choice was that the calculations were very time-consuming and cumbersome. The model was too slow

to allow the exploration of alternative cases and to conduct sensitivity analyses. During the audit,

other, more suitable, qualified codes such as RELAP were identified by the technical specialist.

The AMR indicated that the model used was validated for its intended use. However, the formal model

validation process described in AP 3.10Q had not been accomplished.

4.5.4 Analysis Model Report Drift Degradation Analysis (ANL-EBS-MD-000080, Rev 00)

The AMR defined its objectives as: a) to provide a statistical description of block sizes formed by

fractures around the emplacement drifts; b) to estimate changes in drift profiles resulting from
9

progressive deterioration of the emplacement drifts both with and without backfill; and c) to provide an

estimate of the time required for significant drift deterioration to occur.

The purpose of this AMR, as documented in the development plan (Development for Drift Degradation

Analysis, TDP-EBS-MD-000014 Revision 1, dated September 29,1999) is to analyze the deterioration

of the rock mass surrounding the potential repository emplacement drifts and provide data

(information) to the EBS post-closure performance assessment as well as information for use in the

design of the subsurface openings. The expected output of the analysis would document the

anticipated drift deterioration for the EDA-II design and would provide information for ground-support

design and input to the analyses of waste-package performance. The AMR would also provide input

for two other EBS AMRs: The Physical and Chemical Environment Model, and the Water Distribution

and Removal Model. The development plan did not identify the specific output information that is

required from the AMR in order to satisfy the stated purpose.

The code used for the drift degradation analyses does not have the capability to account for the effects

of thermal and seismic loading on rock fall and drift degradation. The investigators attempted to

include the effects of thermal and seismic loading by reducing the shear strength of fracture surfaces.

However, the inability of the code to account for external loading other than gravity is a fundamental

shortcoming that cannot be remedied by reducing the fracture strength.

As a result, the conclusions in the report regarding the collapsed shape of drifts and the fraction of drift

length that may experience collapse are not adequately supported by the analyses and may need to be

re-examined considering the results from mechanical-analysis codes that explicitly account for the

effects of thermal and seismic loading.

4.6 NRC STAFF FINDINGS

The NRC staff has determined that OQA Audit M&O-ARP-00-006 was effective in determining the level

of compliance of M&O activities associated with the subject AMRs. The NRC staff agrees with the

audit team conclusion that the OCRWM QA program had been satisfactorily implemented and that the

AMRs are adequate considering their early stage of development and their planned continued

development and refinement. However, areas where deficiencies were issued may be an indicator of

ineffective implementation.

While the audit technical specialists generated numerous comments regarding the AMRs and offered

many recommendations to the authors, they appeared to struggle with expressing their technical

comments as findings on the effectiveness of the AMR development/review process and in terms of

assessing the level of quality of the AMRs. The NRC staff recommends that additional efforts be made

to assure that the technical specialists have adequate knowledge to express their findings. It is

recommended that the OQA consider integrating the programmatic and technical portions of the audits

when technical specialists, inexperienced in documenting findings, are part of the audit team. Under

this circumstance, the experienced auditors could coach the less experience technical specialists in

expressing technical shortcomings as programmatic root causes.

The NRC staff recommends that additional effort be made in preparing the technical specialists to be

better able to express their findings. Whenever possible, the DOE audit teams should integrate the

programmatic and technical portions of the audit: 1) to have experienced auditors coach the less

experienced technical specialists; and 2) attempt to express technical shortcomings as programmatic

root causes.

4.6.1 Audit Observer Inquiries

No audit observer inquiries were issued.
10

4.6.2 Closure of Previous NRC Audit Observer Inquiries

No NRC audit observer inquiries were closed during the conduct of this observation.

4.6.3 Open NRC Audit Observer Inquiries (AOIs)

The following NRC Audit Observer Inquiries remain open:

a. Audit Observation Inquiry (AOI) No. OCRWM-ARC-99-015-1, dated September 22, 1999: OQA

agreed to provide information to the NRC on the qualification status and use of the “Waste

Stream Profiles” addressed in the “Design Basis Waste Stream for Interim Storage and

Repository” and the “Waste Quantity, Mix and Throughput Study” documents.

b. AOI No. M&O-ARP-00-02-1, dated November 18, 1999: AP-3.10Q, “Analysis and Modeling”

and the QARD are not specific regarding which calculations/analyses are subject to model

validation and the timing of model validation. M&O Environmental, Safety, and Regional

Programs Office involved with the biosphere AMRs do not appear to have an understanding or

strategy of model validation as it applies to the biosphere AMRs/PMR.

c. AOI No. M&O-ARP-00-02-2, dated November 18, 1999: Documented resolution of individual

comments is not required for checks of analysis and models (AP-3.10Q) and is optional for

reviews of technical products (AP-2.14Q). A lack of documented resolution is inconsistent with

the QARD section 2.2.10(f) which requires that mandatory comments shall be documented and

resolved before approving the document. Note that the audit of the Integrated Site Model

(ARP-99-009) also identified several recommendations concerning the review processes of AP3.10Q and AP-2.14Q.