August 25, 2000

Mr. James H. Carlson, Acting Director

Program Management and Administration

Office of Civilian Radioactive Waste Management

U. S. Department of Energy

1000 Independence Avenue, SW

Washington, DC 20585

SUBJECT: U.S. NUCLEAR REGULATORY COMMISSION’S OBSERVATION AUDIT

REPORT NO. OAR- 00-07, “OBSERVATION AUDIT OF OFFICE OF THE

CIVILIAN RADIOACTIVE WASTE MANAGEMENT, QUALITY ASSURANCE

DIVISION, AUDIT NO. M&O-ARP-00-010"

Dear Mr. Carlson:

I am transmitting the U.S. Nuclear Regulatory Commission’s (NRC’s) Observation Audit Report

No. OAR-00-07 of the U.S. Department of Energy (DOE), Office of Civilian Radioactive Waste

Management (OCRWM), Office of Quality Assurance (OQA), Yucca Mountain Quality

Assurance Division (YMQAD), audit of activities pertaining to the Saturated Zone Flow and

Transport Process Model Report (SZ PMR) performed by the OCRWM Management and

Operating Contractor (M&O). The audit, M&O-ARP-00-010, was conducted on June 21-30,

2000, at the M&O facilities in Las Vegas, Nevada.

The purpose of this performance based audit was to evaluate the effectiveness of the

implementation of the OCRWM QA Program described in the Quality Assurance Requirements

and Description (QARD) and its implementing procedures for the SZ PMR and selected

Analysis Model Reports (AMRs) supporting the SZ PMR.

The NRC staff determined that this audit was effective in identifying potential weaknesses and

recommending improvements in the PMR/AMR process. During the conduct of the audit, both

the audit team and the NRC observers independently reviewed applicable data, analysis

reports, and software to confirm that it was properly qualified. As a result of these activities the

NRC observers determined that the M&O had made substantial progress with respect to the

qualification and verification of data and software which support the PMRs/AMRs. However,

because of the importance of these activities which will sustain site recommendation and the

potential license application, the staff recommends that DOE and the M&O maintain their

emphasis on the timely qualification and verification of data and software which support the

technical adequacy of PMRs/AMRs.

The NRC staff generally agreed with the audit team conclusions, findings, and

recommendations as presented at the Post-audit Conference. Within the areas evaluated, no

deficiencies were identified and the NRC observers concluded that the technical quality and

completeness of the scientific products contained in the SZ PMR and the associated AMRs

indicated an improving trend over the previous six PMR audits performed during the last quarter

of 1999 and the first quarter of 2000.

J. Carlson - 2 -
As discussed in Section 4.6 of the attached report, the NRC observers identified three

concerns pertaining to the clarification of “issue statements” in Appendix A of the SZ PMR,

consideration of alternative conceptual models for the Saturated Zone Flow and Transport

Model, and updating of the status of the regional Saturated Zone Flow Model. These concerns

which complemented the audit team’s findings will be addressed in DOE’s report of the

OCRWM QA Audit M&O-ARP-00-10.

A written response to this letter and the enclosed report is not required. If you have any

questions, please contact Robert M. Latta of my staff at (702) 794-5048.

Sincerely,

/s/

N. King Stablein, Chief (Acting)

High-Level Waste Branch

Division of Waste Management

Office of Nuclear Material Safety

and Safeguards

Enclosure: NRC Observation Audit Report No. OAR-00-07, “Observation Audit of the Office

of Civilian Radioactive Waste Management, Quality Assurance Division, Audit

No. M&O-ARP-00-010"

cc:

R. Loux, State of Nevada S. Frishman, State of Nevada

L. Barrett, DOE/Wash, DC A. Brownstein, DOE/Wash, DC

S. Hanauer, DOE/Wash, DC C. Einberg, DOE/Wash, DC

N. Slater, DOE/Wash, DC R. Dyer, YMPO

S. Brocoum, YMPO R. Clark, YMPO

A. Gil, YMPO S. Echols, Winston & Strawn

G. Dials, M&O J. Bailey, M&O

D. Wilkins, M&O M. Voegele, M&O

D. Bechtel, Clark County, NV B. Price, Nevada Legislative Committee

J. Regan, Churchill County, NV J. Meder, Nevada Legislative Counsel Bureau

G. McCorkell, Esmeralda County, NV E. von Tiesenhousen, Clark County, NV

L. Fiorenzi, Eureka County, NV A. Remus, Inyo County, CA

B. Duke, Lander County, NV J. Pitts, Lincoln County, NV

J. Wallis, Mineral County, NV L. Bradshaw, Nye County, NV

M. Murphy, Nye County, NV J. McKnight, Nye County, NV

B. Ott, White Pine County, NV D. Weigel, GAO

W. Barnard, NWTRB R. Holden, NCAI

A. Collins, NIEC R. Arnold, Pahrump County, NV

J. Lyznicky, AMA R. Clark, EPA

F. Marcinowski, EPA R. Anderson, NEI

R. McCullum, NEI S. Kraft, NEI

J. Kessler, EPRI R. Wallace, USGS

R. Craig, USGS W. Booth, Engineering Svcs., LTD

J. Curtiss, Winston & Strawn
J. Carlson - 2 -

As discussed in Section 4.6 of the attached report, the NRC observers identified three concerns

pertaining to the clarification of “issue statements” in Appendix A of the SZ PMR, consideration of

alternative conceptual models for the Saturated Zone Flow and Transport Model, and updating of

the status of the regional Saturated Zone Flow Model. These concerns which complemented the

audit team’s findings will be addressed in DOE’s report of the OCRWM QA Audit M&O-ARP-00-10.

A written response to this letter and the enclosed report is not required. If you have any questions,

please contact Robert M. Latta of my staff at (702) 794-5048.

Sincerely,

/s/

N King Stablein,(Acting) Chief,

High-Level Waste Branch

Division of Waste Management

Office of Nuclear Material Safety

and Safeguards

Enclosure: NRC Observation Audit Report No. OAR-00-07, “Observation Audit of the Office of

Civilian Radioactive Waste Management, Quality Assurance Division, Audit No.

M&O-ARP-00-010"

DISTRIBUTION:

File Center PUBLIC ACNW LSS CNWRA NMSS r/f HLWB r/f

SWastlerl CGlenn BMabrito WBelke RLatta JHolonich

DWM r/f KHooks TCarter DBrooks

DOCUMENT NAME: S/DWM/HLWB/Rml1/szf&audit.wpd

ADAMS: #ML003745430

OFC HLWB HLWB HLWB

NAME RMLatta:vlm /s/ DJBrooks /s/ NKStablein /s/

DATE 08/25/00 08/25/00 08/25/00

ACNW: YES X NO Delete file after distribution: Yes No

1) This document should/ should not be made available to the PUBLIC _____ _____

(Initials) (Date)

2) This document is related to the HLW program. It should be placed in the LSS. _____ ______
U.S. NUCLEAR REGULATORY COMMISSION

OBSERVATION AUDIT REPORT NO. OAR-00-07

OBSERVATION AUDIT OF THE

OFFICE OF CIVILIAN RADIOACTIVE WASTE MANAGEMENT

QUALITY ASSURANCE DIVISION

AUDIT NO. M&O-ARP-00-010

/s/ 08/25/00 /s/ 08/ 25 /00

Robert M. Latta Latif S. Hamdan

High-Level Waste Branch High-Level Waste Branch

Division of Waste Management Division of Waste Management

/s/ 08/25 /00 /s/ 08/ 25 /00

John W. Bradbury Rodney Weber

High-Level Waste Center for Nuclear Waste Regulatory

Division of Waste Management Analyses

Reviewed and Approved by:

/s/ 08/25/00

N. King Stablein, Chief

Projects and Engineering Section

High-Level Waste Branch

Division of Waste Management
1

1.0 INTRODUCTION

Staff from the U.S. Nuclear Regulatory Commission (NRC) Division of Waste Management and

a contractor from the Center for Nuclear Waste Regulatory Analyses (CNWRA) observed all

aspects of the U.S. Department of Energy (DOE), Office of Civilian Radioactive Waste

Management (OCRWM), Office of Quality Assurance (OQA), Yucca Mountain Quality Assurance

Division (YMQAD) audit of activities pertaining to the Saturated Zone Flow and Transport

Process Model Report (SZ PMR) performed by the OCRWM Management & Operating

Contractor (M&O). The audit, M&O-ARP-00-010, was conducted on June 21–30, 2000, at the

M&O facilities in Las Vegas, Nevada.

The purpose of this audit was to evaluate the implementation of the applicable provisions

contained in the OCRWM Quality Assurance Requirements and Description (QARD), DOE/RW0333P, Revision 9, by evaluating the SZ PMR and selected Analysis Model Reports (AMRs)

supporting the SZ PMR. During the audit, selected AMRs were subjected to a technical

evaluation as well as evaluation to ensure that the applicable programmatic requirements

contained in the QARD and implementing procedures were met.

The NRC staff objective was to gain confidence that the M&O and OQA are properly

implementing the provisions contained in the QARD and the requirements contained in Subpart

G, Quality Assurance, to Part 60, of Title 10 of the Code of Federal Regulations (10 CFR Part

60). Because of the anticipated DOE submittal of the site recommendation (SR) in November

2000, the following observation activities were emphasized: 1) confirming that data, software,

and models supporting SR are properly qualified; and 2) evaluating the progress being made by

DOE and its contractors in meeting the qualification goals for SR.

This report addresses the NRC staff determination of the effectiveness of the OQA audit and the

adequacy of implementation of QARD controls by the M&O in the audited areas of SZ

PMR/AMR development.

2.0 MANAGEMENT SUMMARY

The NRC staff generally agrees with the audit team conclusion’s, findings, and

recommendations. The NRC staff determined that OQA Audit M&O-ARP-00-010 was well

planned and effectively implemented. Audit team members were independent of the activities

they audited and were knowledgeable in the QA and technical disciplines within the scope of the

audit. The audit team members’ qualifications were reviewed and the members were found to

be qualified in their respective disciplines.

The audit team concluded that the OCRWM QA program had been satisfactorily implemented in

the areas evaluated. No deficiencies were identified during the audit. However, twelve

recommendations were identified as improvements to the SZ PMR/AMRs or as enhancements

to the procedures controlling various elements of the modeling reports process.

During the conduct of the audit, both the audit team and the NRC observers reviewed data,

analysis reports, and software within the scope of the audit to confirm that it was properly

qualified. The audit team and the NRC observers determined that elements of the software
2

supporting the AMRs had been properly qualified. The audit team and the NRC observer’s also

determined that certain data, categorized as “accepted data,” were appropriately controlled and

categorized in accordance with the governing procedures.

3.0 AUDIT PARTICIPANTS

3.1 Nuclear Regulatory Commission Observers

Robert Latta Team Leader NRC

John Bradbury Technical Specialist NRC

Latif Hamdan Technical Specialist NRC

Rod Weber QA Specialist CNWRA

3.2 OQA Audit Team

Lester Wagner Audit Team Leader OQA/Quality Assurance Technical

Support Services (OQA/QATSS)

Robert Hartstern Auditor OQA/QATSS

Michael Goyda Auditor OQA/QATSS

Kenneth McFall Auditor OQA/QATSS

Chet Wright Auditor OQA/QATSS

Charles Warren Auditor OQA/QATSS

Keith Kersch Technical Specialist SAIC

Thomas Doe Technical Specialist Management and Technical Services(MTS)

Richard Salness Technical Specialist MTS

3.3 Nevada State Observer

Susan Zimmerman Administrator of Nuclear Waste Project Office,

Technical Programs Agency for Nuclear Projects, State of

Nevada

4.0 REVIEW OF THE AUDIT AND AUDITED ORGANIZATION

This OQA audit of the M&O was conducted in accordance with OCRWM Quality Assurance

Procedure (QAP) 18.2, “Internal Audit Program,” and QAP 16.1Q, “Performance/Deficiency

Reporting.” The NRC staff’ observation of this audit was performed in accordance with NRC

procedure, “Conduct of Observation Audits,” issued October 6, 1989.

4.1 Scope of the Audit

The audit team conducted a limited scope, performance based audit of activities and processes

related to the development of the AMRs supporting the SZ PMR. Audit activities included

evaluation of selected AMRs, software, and associated data. The audit also included review of

the programmatic controls governing the AMRs and technical requirements contained in the

AMRs. The following procedures, and SZ PMR and supporting AMRs were evaluated by the

audit team and the NRC observers during the audit:
3

Procedures

a) AP-2.1Q, “Indoctrination and Training of Personnel,” Revision 0, with Interim Change

Notice (ICN) No. 0

b) AP-2.2Q “Establishment and Verification of Required Educational and Experience of

Personnel,” Revision 0, with ICN No. 0

c) AP-2.13Q, “Technical Product Development Planning,” Revision 0, with ICN No.3

d) AP-2.14Q, “Review of Technical Products,” Revision 0, with ICN No. 1

e) AP-2.15Q, “Work Package Planning Summaries,” Revision 0, ICN No.1

f) AP-3.4Q, “Level 3 Change Control,” Revision 1, ICN No.3

g) AP-3.10Q, “Analysis and Models,” Revision 2, with ICN No. 2

h) AP-3.11Q, “Technical Reports” Revision 1, with ICN No. 1

i) AP-3.14Q, “Transmittal of Input” Revision 0, with ICN No. 0

j) AP-3.15Q, “Managing Technical Product Inputs,” Revision 1, with ICN No. 1

k) AP-3.17Q, “Impact Reviews,” Revision 0, with ICN No. 0

l) AP-SI.1Q, “Software Management,” Revision 2, with ICN No. 4

m) AP-SIII-1Q, “Scientific Notebooks,” Revision 0, with ICN No. 1

n) AP-SIII.2Q, “Qualification of Unqualified Data and the Documentation of Rationale for

Accepted Data,” Revision 0, with ICN No. 2

o) AP-SIII.3Q, “Submittal and Incorporation of Data to the TDMS,” Revision 0, with ICN No.

3

p) AP-SV.1Q, “Control of the Electronic Management of Data,” Revision 0, with ICN No. 1

q) QAP-18.1, “Auditor Qualification,” Revision 6, with ICN No. 0

Process Model Report

a) TDR-NSB-HS-000001, “Saturated Zone Flow and Transport,” Revision 00

Analysis Model Reports

a) ANL-NBS-HS-000033, “Hydrogeologic Framework Model for the Saturated-Zone SiteScale Flow and Transport Model,” Revision 00E
4

b) Analysis Model Report, Calibration of the Site-Scale Saturated Zone Flow Model (MDLNSB-HS-000011, Rev 00F)

c) ANL-NBS-HS-00003, “Input and Results of the Base Case Saturated Zone Flow and

Transport Model for TSPA,” Revision 00

4.2 Conduct and Timing of the Audit

The audit was performed effectively and the OQA audit team demonstrated a sound knowledge

of the applicable M&O and DOE programs and procedures. Audit team members conducted

thorough interviews, they challenged responses when appropriate and they effectively employed

their detailed audit checklists. The NRC staff believes the timing of the audit was appropriate for

the auditors to evaluate ongoing SZ PMR activities. However, the audit team was limited in their

ability to confirm that data supporting the AMRs had been properly qualified because only a

small number of the associated data sets had been qualified in accordance with the

requirements of Procedure AP-3.15Q.

The DOE audit team and NRC observers caucused at the end of each day. Meetings between

the audit team and M&O management (with the NRC observers present) were also held each

morning to discuss the current audit status and preliminary findings.

4.3 Audit Team Qualification and Independence

The qualifications of the audit team leader and the OQA audit team were reviewed for accuracy

and completeness in accordance with the requirements of Procedure QAP 18.1, “Auditor

Qualification”. The NRC staff review included an examination of the training, education,

experience, and annual evaluation records of the audit team members. As a result of these

reviews, one item was identified and discussed with the Auditor Certification Coordinator.

Specifically, paragraph 5.1.3 b, of Procedure QAP 18.1, states that the Certification Coordinator

shall prepare a memorandum to file attesting to the completion of required training for the

prospective auditor. However, the records for three of the more experienced auditors did not

contain the specified memorandum. The significance of this issue was determined to be minor,

in that the requisite training for these individuals had been accomplished and the only

discrepancy was that the individual’s files had not been updated to reflect this condition.

Subsequent to the identification of this issue, the Certification Coordinator agreed to review the

qualification records and take the necessary administrative action to address this oversight.

4.4 Examination of Quality Assurance Elements

As defined in the audit schedule, the OQA programmatic and technical audit activities were

conducted simultaneously using sub-audit teams consisting of a technical specialist and a QA

auditor. The limited scope audit focused on the QA elements closely associated with the

development of the AMRs. The NRC observation team evaluated the audit team’s review of the

following QA elements.

4.4.1 AP-2.13Q “Technical Product Development Planning”

The auditors reviewed technical development plans and work product planning sheets applicable

to the subject AMRs. In addition, the auditors reviewed the methodology for the product
5

development, including the tracking of unresolved issues [inputs requiring qualification, to be

verified (TBV), etc.]. No significant issues were identified within this area of review.

4.4.2 AP-SI.1Q “Software Management”

Software controls associated with the SZ PMR/AMRs were discussed during each of the

technical interviews. The auditors reviewed qualification documentation and determined that the

requirements of the software management procedure had been met. The Calibration of the SiteScale Saturated Zone Flow Model draft AMR was found to be based on an older version of the

Regional Saturated Zone flow model. The audit team recommended that a discussion of the

impact on the AMR and justification for use of the model be included in the AMR. The software

SZ-CONVOLUTE was found to be called out twice in the Input and Results of the Base Case

Saturated Zone Flow and Transport Model for TSPA AMR. However, no information for this

program was included in the reference listing. Accordingly, it was recommended that a

reference be included in the next revision and that it indicate where the software can be found.

4.4.3 AP-3.15Q “Managing Technical Product Inputs”

Each of the AMRs examined included document input reference sheets that list the inputs to and

references cited in the AMR. The document input reference sheets also identify the status of the

input, [e.g. qualified, to be verified (TBV)]. The NRC observers examined the TBV status and

determined that it included the appropriate statements in accordance with the Analysis/Model

Documentation Outline. For the Calibration of the Site-Scale Saturated Zone Flow Model AMR,

Rev. 00E, it was noted by the audit team that there are weaknesses in the statements related to

TBV input effects on the model. Accordingly, it was recommended that future revisions provide

clarification within this area.

4.4.4 AP-3.10Q “Analysis and Models”

Procedure AP-3.10Q was used by the audit team to evaluate the activities covered during the

audit. By definition, this procedure applies to activities pertaining to the development,

documentation, checking, review, approval, and revision of analyses or models, and the

calibration, validation, or use of models to support scientific, engineering, or performanceassessment work activities.

Although the audit team generally concluded that the requirements of Procedure AP-3.10Q have

been appropriately implemented, one item related to improved documentation was identified.

Specifically, the audit team determined that the SZ PMR and the corresponding UZ PMR had

been developed in parallel. This parallel development resulted in the use of input flux in the SZ

PMR that is based on a 1997 UZ database which was subsequently updated in 1999. Although

the revised data were properly used in the development of the UZ PMR they were not

incorporated into the SZ PMR. The M&O staff indicated that an analysis was conducted to

assess the impact of the new data, and that this analysis indicated that the impact on the SZ

PMR was insignificant. However, the audit team noted that there was no documentation of the

impact analysis in the SZ PMR.

The NRC technical observers also noted that alternative conceptual models for SZ flow paths

and potentiometric heads including the water table were neither identified nor analyzed as
6

required by Procedure AP-3.10Q (i.e., in Attachment 1, Section 6: Analysis/Model

Documentation Outline).

As established during the Post-audit Conference, these issues were analogous to the findings of

the audit team, and they will be addressed in DOE’s report of the OCRWM QA Audit M&O-ARP00-10.

4.4.5 AP-2.14Q “Review of Technical Products”

The SZ PMR and two of the AMRs evaluated during this audit were subjected to the technical

review process defined in Procedure AP-2.14Q. These AP-2.14Q reviews were performed by

the M&O’s Data/Software Qualification Department which is external to the organizations that

prepared the SZ PMR and AMRs. However, ANL-NBS-HS-00003, “Input and Results of the

Base Case Saturated Zone Flow and Transport Model for TSPA” Revision 00, had not

undergone the AP-2.14Q review process.

The NRC observers also examined the process controls for the resolution of to-be-verified (TBV)

data used as direct input to AMRs and PMRs, implicit to the review of technical products. As

described in Procedure AP-3.15Q, the term “TBV”, is used to identify information which is

preliminary in nature, that needs to be re-evaluated and/or requires confirmation. The

procedural controls of AP-3.15Q, which are applicable to the M&O, the National Laboratories

and U.S. Geological Survey (USGS) establish the organizational responsibilities and processes

required for the acquisition, tracking, and status of the technical product inputs necessary for the

development of Yucca Mountain Site characterization project documents. This procedure also

describes the necessary actions to resolve TBVs and to-be-determined (TBDs) and the

administrative controls to track incomplete reference checks on inputs used in approved

technical products.

The NRC observers held discussions with cognizant individuals in the Data/Software

Qualification Department and reviewed selected data tracking number (DTN) sets in order to

gain insights into the verification methodology to resolve TBVs.

As a result of these discussions and review activities the NRC observers ascertained that

approximately 66% of the data supporting the site recommendation and the potential license

application had been qualified and that approximately 85% of the data in the Document Input

Reference System (DIRS) had been verified. Based on these values the NRC observers

determined that the M&O had made substantial progress with respect to the

qualification/verification of data and software which support the AMRs/PMRs. However,

because of the importance of these activities which will sustain site recommendation and the

potential license application, the staff recommends that DOE and the M&O maintain their

emphasis on the timely qualification and verification of data and software which support the

technical adequacy of the PMRs/AMRs.

4.6 Examination of Technical Activities

Technical specialists on the audit team performed detailed checks of the technical adequacy of

the subject SZ PMR/AMRs. Technical observers from NRC observed the audit of the these

activities.
7

The technical specialist qualifications (resumes) were reviewed. As a result of these reviews it

was determined that appropriate educational backgrounds, training, and experience for these

individuals had been documented.

The technical specialists on the audit team evaluated activities and processes supporting the

development of the SZ PMR. The technical specialists used a combination of technical

questioning and programmatic compliance checks to verify AMR technical adequacy and QA

program effectiveness. The technical activities were evaluated using three evaluation criteria

pertaining to transparency; traceability; and defensibility.

The audit checklist included items pertaining to the SZ PMR and three supporting AMRs, which

were pre-selected by the auditors from a total of 13 AMRs supporting the SZ PMR. In addition

to auditing the pre-selected AMRs, the technical specialists and NRC technical observers also

examined information and analyses provided in one other draft AMR. This was a draft AMR

pertaining to Water-Level Data Analysis for the Saturated Zone Site-Scale Flow and Transport

Model (MDL-NSB-HS-000034, Rev C), which provided information that supported the

Hydrogeologic Framework Model (HFM) and Calibration of the Site-Scale Model AMR.

Based on these reviews the NRC observers concurred with the DOE audit team that the

technical content of the PMR/AMRs satisfied the audit evaluation criteria. However, the NRC

technical staff did identify three concerns related to the SZ PMR which are described in sections

4.6.1 through 4.6.4, of this report. These concerns which complemented the audit team’s

findings will be addressed in DOE’s report of the OCRWM QA Audit M&O-ARP-00-10.

4.6.1 Process Model Report, Saturated Zone Flow and Transport (TRD-NBS-HS000001,Rev.00)

A total of 14 items pertinent to the SZ PMR were included in the audit checklist, and evaluated

by the auditors. The following technical subjects were addressed: input data for the SZ site-scale

model, unit breakthrough curves, unsaturated zone (UZ) mass flux, input to the convolution

integral program, overview of the results of the TSPA calculation, uses of UZ expert elicitation on

groundwater fluxes, code verification (FEHM, particle tracker), timing on inputs, and QA issues.

The technical specialists, on the audit team, reviewed the pertinent information concerning the

above listed items. Based on the results of these reviews the technical specialists were

generally satisfied that the SZ PMR met the audit evaluation criteria. The audit team further

determined that the SZ PMR and the corresponding UZ PMR were developed in parallel, which

resulted in using input flux in the SZ PMR that was based on a 1997 database that had

subsequently been updated in 1999. The audit team ascertained that the new data were

appropriately used in the development of UZ PMR, but they were not utilized in the SZ PMR.

The M&O staff stated that an analysis was conducted to assess the impact of the new data, and

that the impact analysis indicated that the effect on the SZ PMR was insignificant. However, the

audit team noted that there was no objective evidence of this analysis in the SZ PMR.

The NRC technical observers also noted that numerous comments on the site-scale and

regional SZ flow models provided by outside reviewers had not been appropriately addressed in

the SZ PMR. Abstractions of reviewer comments on the SZ flow models were provided in the

SZ PMR (i.e., comments by the Nuclear Waste Technical Review Board, DOE’s Peer Review

Panel, and DOE’s SZ Expert Elicitation Panel are provided in Appendix A; and NRC staff
8

comments are provided in Appendix B of the PMR). However, many of the responses to the

comment abstractions were unclear or non-responsive, and it appeared that many of the

reviewers concerns were neither resolved nor mitigated.

The DOE audit team acknowledged the NRC technical observer’s concerns which were

complementary to the audit team findings and agreed to address these items in their report of

the OCRWM QA Audit M&O-ARP-00-10.

4.6.2 Analysis Model Report, Hydrogeologic Framework Model for the Saturated-Zone

Site-Scale Flow and Transport Model, (ANL-NBS-HS-000033, Rev 00E)

A total of 23 audit checklist items, pertinent to this AMR were evaluated by the auditors. The

following technical subjects were addressed: data storage and control, scientific notebooks,

correlation of borehole data and geologic sections, documentation of calculations and methods,

description and documentation of hybrid gridding techniques, plans for qualification of input data,

process of inputting borehole logs and geologic maps into the Hydrogeologic Framework Model

(HFM), use of the Geologic Framework Model in the HFM, information on geologic cross

sections incorporated in the HFM, timeliness of input data update and incorporation of updated

HFM model in other models, measures for data sufficiency, method of integration of structural

data into the HFM, use of input data to create Stratamodel files, stacking number sequences

and consistency between different versions of the HFM, process of leveling and digital

referencing to map traces, construction steps of the 3-D HFM and documentation of construction

steps in procedure and in scientific notebooks, updating of the potentiometric surface in the

HFM, and the processes of clipping versus extrapolation where data do not exist.

The audit team requested and was provided information on each of the above listed items.

Based on the review of this information the technical specialists were generally satisfied that the

this AMR met the audit evaluation criteria.

The NRC technical observers noted that the water table, which constitutes the upper boundary

for the HFM has a large measure of uncertainty in that it is based on water level measurements

in different aquifer units including confined aquifers. Although the AMR acknowledges that there

are alternative conceptual models for the “water table”, the NRC technical observers were

concerned that these alternative conceptual models were neither identified nor analyzed.

Examination of another AMR (i.e., Water-Level Data Analysis for the Saturated Zone Site-Scale

Flow and Transport Model, S-000034), which was cited by the M&O staff as the source for the

water table model, confirmed that alternative conceptual models of the water level were not

identified or analyzed.

The DOE audit team acknowledged the NRC technical observer’s concern which was

complementary to the audit team findings and agreed to address this item in their report of the

OCRWM QA Audit M&O-ARP-00-10.

4.6.3 Analysis Model Report, Calibration of the Site-Scale Saturated Zone Flow Model

(MDL-NSB-HS-000011, Rev 00F)

A total of 17 audit checklist items, pertinent to this AMR, were evaluated by the auditors. The

following technical subjects were addressed: completeness of work including references,

checking the results of the NETPATH code, proper referencing of software codes, grid
9

resolution, FEHM boundary condition macro, local recharge, sensitivity of estimated parameter

values, analysis of weighted residuals, conversion of FEHM code output to TECPLOT code

input, effective continuum approach, anisotropy representation in the SZ model, HFM

representation in the SZ model, infiltration map and linkage of SZ and UZ models, calibration

goal, calibration error, importance of wells in the calibration.

The audit team requested and was provided information on each of the above listed items.

Based on the review of this information the technical specialists were generally satisfied that this

AMR met the audit evaluation criteria.

The NRC technical observers were concerned that the use of currently available regional SZ

flow model in the calibration of the site-scale model might not be appropriate. They pointed out

that the current regional model is not in active use, and that if a regional model is to be used in

the calibration or validation of the site-scale model, the updated version of the regional model,

currently under development by the U.S.G.S., was preferable. The NRC technical observers

noted further that the status of the regional model should to be clearly documented in the AMR.

The DOE audit team acknowledged the NRC technical observer’s concerns which were

complementary to the audit team findings and agreed to address these items in their report of

the OCRWM QA Audit M&O-ARP-00-10.

4.6.4 Analysis Model Report, Input and Results of the Base Case Saturated Zone Flow

and Transport Model for TSPA, (ANL-NBS-HS-00003, Revision 00)

A total of 20 audit checklist items, pertinent to this AMR, were evaluated by the auditors. The

following technical areas were specifically addressed: plan and schedule to qualify unqualified

software, documentation and referencing of software testing, time conversion factors, relative

mass flux, “SZ-CONVOLUT” computer code documentation, validation of software routines,

documentation of data transfer from other AMRs and from other sources, explanation for the

assumption that the point source for radionuclide transport in the SZ is conservative,

assumptions pertaining to the one-dimensional radionuclide transport modeling, rationale for

using 100 realizations, applicability of the volcanic aquifer flux to the entire flow path from the

repository to the biosphere, controls to avoid propagation of data errors and procedures to check

for errors, evaluation of sensitivity of parameters in the stochastic representation in the model,

how the SZ model is updated as the UZ site-scale model data changes, verification process of

the flux computations, derivation of alluvium uncertainty zone, horizontal anisotropy derivation,

determination of impact of climate changes on flow paths, and sources of parameter values

used for model comparison.

The audit team requested and was provided information on each of the above listed items.

Based on the review of this information the audit team was generally satisfied that the AMR met

the audit evaluation criteria.

The NRC technical observers noted that field tests simulating radionuclide transport in the

saturated zone have been conducted in the C-well complex. The scale of these tests was on the

order of 30 meters, whereas the size of the grid blocks used in the Total System Performance

Assessment (TSPA) is 500 meters on a side. Homogeneous material properties are assigned to

individual hydrogeologic units in the TSPA. With the possibility that transport parameters are

scale dependent, the NRC technical observers were concerned that the data collected in smallscale field experiments may not support the parameter ranges used in TSPA., where the grid
10

size is an order of magnitude larger than the field tests. The NRC technical observers

recommended that future revisions to the AMR address this issue and clearly delineate how the

field tests support the SZ transport model.

The DOE audit team acknowledged the NRC technical observer’s concerns which were

complementary to the audit team findings and agreed to address these items in their report of

the OCRWM QA Audit M&O-ARP-00-10.

4.7 NUCLEAR REGULATORY COMMISSION STAFF FINDINGS

The NRC staff has determined that OQA Audit M&O-ARP-00-010 was effective in determining

the level of compliance of M&O activities associated with the subject AMRs. The NRC staff

agrees with the audit team conclusion that the OCRWM QA program had been satisfactorily

implemented. No deficiencies were identified during the audit and the NRC staff concluded that

the technical quality and completeness of the scientific products in the SZ PMR and the

associated AMRs indicated an improving trend over the products evaluated in the previous six

PMR audits conducted in the last quarter of 1999 and the first quarter of 2000.

4.7.1 Closure of Previous NRC Audit Observer Inquiries

a) AOI No. OCRWM-ARC-99-015-1, dated September 22, 1999, requested additional

information and clarification on the qualification status and use of the “Waste Stream

Profiles” addressed in the “Design Basis Waste Stream for Interim Storage and

Repository” and the “Waste Quantity, Mix and Throughput Study” documents. The

response provided in DOE’s Letter to the NRC (C.W. Reamer) from DOE (R.W. Clark),

“U.S. Nuclear Regulatory Commission (NRC) Auditor Observer Inquiries,” dated April 4,

2000, provides an acceptable response to this inquiry.

b) AOI No. M&O-ARP-00-02-1, dated November 18, 1999, identified that Procedure AP3.10Q, “Analysis and Modeling” and the QARD are not specific regarding which

calculations/analyses are subject to model validation and the timing of model validation,

and that the M&O Environmental, Safety and Regional Programs Office, involved with the

Biosphere AMRs, do not appear to have an understanding or strategy for model

validation as it applies to the biosphere AMRs/PMRs. The response provided in DOE’s

Letter to the NRC (C.W. Reamer) from DOE (R.W. Clark), “U.S. Nuclear Regulatory

Commission (NRC) Auditor Observation Inquires,” dated April 4, 2000, provides an

acceptable response to this inquiry.

c) AOI No. M&O-ARP-00-02-2, dated November 18, 1999, was closed during the conduct of

this audit/observation. This AOI documented that the resolution of individual comments

was not required for checks of analysis and models (AP-3.10Q) and that it was optional

for review of technical products (AP-2.14Q). However, the lack of a documented

resolution to these issues is inconsistent with the QARD, Section 2.2.10 (F), which

requires that mandatory comments shall be documented and resolved before approval of

the document.

Based on the review of DOE’s Letter to the NRC (C.W. Reamer) from DOE (R.W. Clark),

dated April 4, 2000, it was determined that Procedure AP-3.10Q has been revised to

require documentation of comment resolution in accordance with Procedure AP-2.14Q
11

and that the staff responsible for the implementation of these procedures had been

appropriately trained. Therefore, the NRC observers concluded that appropriate

corrective actions had been implemented to resolve this AOI.