January 9, 2006

Mr. Mark H. Williams, Director

Office of License Application and Strategy

Office of Repository Development

U. S. Department of Energy

1551 Hillshire Drive

Las Vegas, NV 89134-6321

SUBJECT: U.S. NUCLEAR REGULATORY COMMISSION OBSERVATION AUDIT

REPORT NO. OAR-05-05, OBSERVATION AUDIT OF BECHTEL SAIC

COMPANY, LLC INTERNAL AUDIT BQAP-BSC-05-07

Dear Mr. Williams:

Enclosed is the U.S. Nuclear Regulatory Commission (NRC) Observation Audit Report 

(OAR-05-05) of the Bechtel SAIC Company, LLC (BSC), internal performance-based audit

BQAP-BSC-05-07, conducted on August 15 - 31, 2005, at the U.S. Department of Energy

(DOE) facilities in Las Vegas, Nevada, and at Lawrence Livermore National Laboratories

(LLNL) in Livermore, California.

The BSC audit team assessed the effectiveness and implementation of the quality assurance

program requirements applicable to scientific investigations supporting the waste package and

drip shield degradation models. At the conclusion of the audit, the BSC audit team presented

its findings at an exit briefing with DOE and BSC management and issued an audit report,

dated October 7, 2005. Based on the results of its evaluations, the BSC audit team identified

several condition adverse to quality, including problems with the documentation, traceability and

reproducibility of the Thermal-Hydrological-Chemical (THC) Seepage Model. With the

exception of the THC Seepage Model, the BSC audit team’s conclusion was that the overall

performance of audited activities was satisfactory. In addition, the BSC audit team found that

LLNL satisfactorily and effectively implemented the examined sections of the DOE Quality

Assurance Requirements and Description (QARD) document, Revision 16.

NRC conducted an audit observation of BSC’s audit in accordance with NRC Manual

Chapter 2410, “Conduct of Observation Audits.” The NRC team observed the BSC audit team

members performing the audit, reviewed the same documentation as the BSC audit team, and

attended various interviews conducted by the BSC audit team. Regarding the acceptability of

the models, NRC found a number of issues that are identified as either audit observation

inquiries or weaknesses, in this report, that we believe the BSC audit team should have

considered, as part of deciding that the models were adequate. In addition, contrary to the

findings of the BSC audit team, the NRC observers concluded that LLNL did not effectively

implement the control of maintenance and test equipment and corrective action requirements of

DOE’s QARD, related to the experiments conducted to study the effects of deliquescence on

the corrosion rates of the waste package. Specifically, LLNL corrosion experiments were

conducted at temperature ranges where the vendor had not certified the relative humidity

probes nor calibrated them before use. Although these deficiencies were identified in Condition
 M. Williams - 2 -

Reports, LLNL continued to use the subject relative humidity probes in corrosion experiments, a

condition which apparently rendered the data obtained from the experiments unusable. NRC

brought this matter to the attention of the BSC audit team; however, it did not document the

condition of its audit reports or effectively reflect it in its assessment of LLNL’s compliance with

the QARD requirements. As a result of these issues, NRC has concluded that the BSC audit

was not effective in identifying, documenting, and alerting BSC and DOE management to the

significance of the issues, at LLNL, which were in non-compliance with the QARD

requirements. Items the NRC observers identified as involving non-compliance with the QARD

are documented as deficiencies in this report.

A written response to this letter and the enclosed report is not required; however, a response is

requested to the open Audit Observer Inquiries identified in Section 5.2 of the enclosed report. 

The staff will continue to interface with DOE and follow the action that you take to address the

issues identified during this audit. If you have any questions, please contact Thomas Matula at

301-415-6700.

Sincerely,

/RA/

Elmo E. Collins, Deputy Director

Licensing and Inspection Directorate

Division of High-Level Waste Repository Safety

Office of Nuclear Material Safety

 and Safeguards

Enclosure: U.S. Nuclear Regulatory 

Commission Observation 

Audit Report OAR-05-05, 

“Observation Audit of Bechtel

SAID Company, LLC, Audit of 

Scientific Investigation, 

Waste Package, and Drip 

Shield Degradation,” 

BAD.-BSC-05-07

cc: See Attached List
Letter To: M. Williams From: E. Collins dated: January 9, 2006 

cc: 

A. Kalt, Churchill County, NV A. Elzeftawy, Las Vegas Paiute Tribe 

R. Massey, Churchill/Lander County, NV J. Treichel, Nuclear Waste Task Force

I. Navis, Clark County, NV W. Briggs, Ross, Dixon & Bell

E. von Tiesenhausen, Clark County, NV R. Murray, DOE/ORD

G. McCorkell, Esmeralda County, NV G. Runkle, DOE/Washington, D.C.

R. Damele, Eureka County, NV C. Einberg, DOE/Washington, D.C.

L. Marshall, Eureka County, NV S. Gomberg, DOE/Washington, D.C.

A. Johnson, Eureka County, NV W. J. Arthur, III , DOE/ORD

L. Klusmire, Inyo County, CA R. Dyer, DOE/ORD

M. Yarbro, Lander County, NV J. Espinoza, GAO

J. Donnell, DOE/ORD A. Gil, DOE/ORD

M. Baughman, Lincoln County, NV W. Boyle, DOE/ORD

L. Mathias, Mineral County, NV D. Brown, DOE/OCRWM

D. Swanson, Nye County, NV S.A. Wade,, DOE/ORD

M. Henderson, Cong. J. Gibbon’s Office C. Hanlon, DOE/ORD

D. Hammermeister, Nye County, NV T. Gunter, DOE/ORD

M. Simon, White Pine County, NV A. Benson, DOE/ORD

J. Ray, NV Congressional Delegation N. Hunemuller, DOE/ORD

D. Cornwall, NV Congressional Delegation P. Harrington, OPM&E

T. Story, NV Congressional Delegation M. Mason, BSC

R. Herbert, NV Congressional Delegation S. Cereghino, BSC

M. Murphy, Nye County, NV B. Gattoni, Burns & Roe

R. Lambe, NV Congressional Delegation E. Mueller, BSC

K. Kirkeby, NV Congressional Delegation T. Feigenbaum, BSC

R. Loux, State of NV D. Beckman, BSC/B&A

S. Frishman, State of NV L. Rasura, Lincoln County, NV

S. Lynch, State of NV J. Kennedy, Timbisha Shoshone Tribe

P. Guinan, Legislative Counsel Bureau B. Durham, Timbisha Shoshone Tribe

R. Clark, EPA R. Arnold, Pahrump Paiute Tribe

R. Anderson, NEI J. Birchim, Yomba Shoshone Tribe
cc: (Continued)

R. McCullum, NEI R. Holden, NCAI

S. Kraft, NEI C. Meyers, Moapa Paiute Indian Tribe

J. Kessler, EPRI R. Wilder, Fort Independence Indian Tribe

D. Duncan, USGS D. Vega, Bishop Paiute Indian Tribe

R. Craig, USGS Egan, Fitzpatrick, Malsch, PLLC

W. Booth, Engineering Svcs, LTD J. Leeds, Las Vegas Indian Center

C. Marden, BNFL Inc. J. C. Saulque, Benton Paiute Indian Tribe

A. Robinson, Robinson-Seidler C. Bradley, Kaibab Band of Southern Paiutes

P. Thompson, Duckwater Shoshone Tribe R. Joseph, Lone Pine Paiute-Shoshone Tribe

T. Kingham, GAO L. Tom, Paiute Indian Tribes of Utah

D. Feehan, GAO E. Smith, Chemehuevi Indian Tribe

E. Hiruo, Platts Nuclear Publications D. Buckner, Ely Shoshone Tribe

G. Hernandez, Las Vegas Paiute Tribe V. Guzman, Walker River Paiute 

K. Finfrock, NV Congressional Delegation D. Eddy, Jr., Colorado River Indian Tribes

P. Johnson, Citizen Alert H. Jackson, Public Citizen

M. Williams, DOE/ORD J. Wells, Western Shoshone National Council

J. Williams, DOE/Washington, DC D. Crawford, Inter-Tribal Council of NV

J. Bacoch, Big Pine Paiute Tribe of the

Owens Valley

I. Zabarte, Western Shoshone National

Council

M. Plaster, City of Las Vegas S. Devlin

D. Wilson, Sen. Reid’s Office G. Hudlow

L. Lehman, T-REG, Inc. D. Irwin, Hunton & Williams

W. Barnard, NWTRB P. Golan, DOE

M. Urie, DOE M. Rice, Lincoln County, NV

J. Saldarini, BSC G. Hellstrom, DOE

B. Sagar, CNWRA
M. Williams - 2 -

Reports, LLNL continued to use the subject relative humidity probes in corrosion experiments, a

condition which apparently rendered the data obtained from the experiments unusable. NRC

brought this matter to the attention of the BSC audit team; however, it did not document the

condition of its audit reports or effectively reflect it in its assessment of LLNL’s compliance with

the QARD requirements. As a result of these issues, NRC has concluded that the BSC audit

was not effective in identifying, documenting, and alerting BSC and DOE management to the

significance of the issues, at LLNL, which were in non-compliance with the QARD

requirements. Items the NRC observers identified as involving non-compliance with the QARD

are documented as deficiencies in this report.

A written response to this letter and the enclosed report is not required; however, a response is

requested to the open Audit Observer Inquiries identified in Section 5.2 of the enclosed report. 

The staff will continue to interface with DOE and follow the action that you take to address the

issues identified during this audit. If you have any questions, please contact Thomas Matula at

301-415-6700.

Sincerely,

/RA/

Elmo E. Collins, Deputy Director

Licensing and Inspection Directorate

Division of High-Level Waste Repository Safety

Office of Nuclear Material Safety

 and Safeguards

Enclosure: U.S. Nuclear Regulatory 

Commission Observation 

Audit Report OAR-05-05, 

“Observation Audit of Bechtel

SAID Company, LLC, Audit of 

Scientific Investigation, 

Waste Package, and Drip 

Shield Degradation,” 

BAP.-BSC-05-07

cc: See Attached List.

DISTRIBUTION: ACNW PUBLIC OSRs CNWRA RBrient

BSpitzberg JParrot RLatta VEverett HLWRS Staff SN

ML053470527

OFC HLWB HLWB Tech Editor Region IV HLWB

NAME TMatula MBailey EKraus DBSpitzberg FBrown

DATE 12/19/05 12/19/05 12/19/05 12/20/05 12/30/05

OFC OGC HLWB

NAME JMoore -

MZobler for

ECollins

DATE 12/22/05 1/09/06

 OFFICIAL RECORD COPY
U.S. NUCLEAR REGULATORY COMMISSION OBSERVATION AUDIT REPORT 

NO. OAR-05-05,

“OBSERVATION AUDIT OF BECHTEL SAIC COMPANY, LLC,

AUDIT OF SCIENTIFIC INVESTIGATION,

WASTE PACKAGE, AND DRIP SHIELD DEGRADATION,” BQAP-BSC-05-7

 /RA/ signed by R. Latta 12/08/05 /RA/ signed by R. Weber 12/09/05

Robert Latta Rodney M. Weber

Senior Site Representative Center for Nuclear Waste Regulatory

Fuel Cycle/Decommissioning Branch Analyses

Division of Nuclear Materials Safety

Region IV

 /RA/ signed by A. Csontos 12/08/05 /RA/ signed by L. Yang 12/09/05

Aladar Csontos Lietai Yang

Engineering Section Center for Nuclear Waste Regulatory

Technical Review Directorate Analyses

Division of High-Level Waste

 Repository Safety

 /RA/ signed by V. Everett 12/20/05

Vincent Everett

Fuel Cycle/Decommissioning Branch

Division of Nuclear Materials Safety

Region IV

Reviewed and Approved by:

 /RA/ signed by E. Collins 1/9/06

Elmo Collins, Chief

Licensing and Inspection Directorate

Division of High-Level Waste Repository Safety

Office of Nuclear Material Safety and Safeguards
1.0 INTRODUCTION

On August 15 – 31, 2005, staff from the U.S. Nuclear Regulatory Commission (NRC) Division

of High-Level Waste Repository Safety, NRC Region IV, and the Center for Nuclear Waste

Regulatory Analyses (CNWRA) observed the Bechtel SAIC Company, LLC (BSC) audit

BQAP–BSC–05–07 of “BSC Waste Package Degradation Investigations and Analyses.” The

BSC audit team, comprised of quality assurance (QA) personnel and technical specialists,

conducted the audit at the U.S. Department of Energy (DOE) facilities in Las Vegas, Nevada,

and at the Lawrence Livermore National Laboratory (LLNL) facilities in Livermore, California. 

The purpose of this audit was to evaluate whether scientific investigations provided input, into

the analysis and models for the waste package and drip shield degradation studies, that

resulted in technically sound and defensible results. The audit included evaluations of the

implementation of the Office of Civilian Radioactive Waste Management Quality Assurance

Requirements and Description (QARD), DOE/RW–0333P, Revision 16, and technical

evaluation of work against NUREG-1804, “Yucca Mountain Review Plan, Acceptance Criterion

2,” Section 2.2.1.3.1.3. The NRC observers conducted their activities in accordance with NRC

Manual Chapter 2410, “Conduct of Observation Audits,” while observing the audit. In

accordance with the guidance in Manual Chapter 2410, the NRC observers were to: (i) evaluate

the effectiveness of the BSC audit team to identify deficiencies in the QA program of the

organizations being audited; and (ii) to evaluate the implementation of the QA program by the

organization being audited. 

2.0 MANAGEMENT SUMMARY

NRC observers conducted an audit observation of the BSC audit team activities to assess the

effectiveness of the audit and to evaluate the implementation of the QA program by the

organizations being audited. The BSC audit team included the NRC observers in all activities

related to the audit and allowed full access to the documents being reviewed and the interviews

conducted. NRC observers were allowed the necessary freedom to make an independent

evaluation of the effectiveness of the audit and the implementation of the QA program

regarding the corrosion studies reviewed during this audit. The observers reviewed the

qualifications of the Audit Team Leader, auditors, and Technical Specialists and determined

that they were qualified by education, experience, and training and were independent of the

areas reviewed.

The NRC observers provided periodic feedback to the BSC audit team related to issues

identified during the audit and conducted a post-audit review, with the BSC audit team, of the

more significant concerns identified by the NRC observers. These concerns are described in

this report and focus on several key issues that are summarized in Section 5.1, “NRC

Observation Summary.” The areas of concern included the inadequate implementation of the

QARD related to the work at LLNL for the deliquescence experiments that used uncalibrated

humidity probes, the referencing of cancelled documents in the analysis model reports, and the

inadequate surface cleaning of several samples used to determine corrosion rates. The issues

related to the adequacy of the audit include concerns with the last-minute reductions in the

audit scope and the lack of technical representation on the audit team that went to LLNL.

As a result of these issues, NRC does not agree with the conclusion reached by BSC, in its

audit report issued October 7, 2005, that LLNL was satisfactorily and effectively implementing

the QARD for the areas reviewed in this audit. NRC agreed with the BSC audit conclusions

regarding the problems with documentation, traceability, and reproducibility with the Thermal-
Hydrological-Chemical (THC) Seepage Model.

3.0 AUDIT PARTICIPANTS

BSC Auditors/Technical Specialists

Judith Shipman, Audit Team Leader

Carl Wright, Auditor

Ajulena Barnes, Auditor

Sandra Castro, Auditor

Thomas Esper, Auditor

Bob Habbe, Auditor

Stanley Levine, Auditor

Stephen Schuermann, Auditor

Stanley Howard, Ph.D., Technical Specialist, Metallurgy/Corrosion

Kathryn Johnson, Ph.D., Technical Specialist, Geochemistry

NRC Observers/Technical Specialists

Robert Latta, NRC, Team Leader

Vincent Everett, NRC, Senior Inspector

Aladar Csontos, NRC, Ph.D., Technical Specialist

Rodney Weber, CNWRA, QA Specialist

Lietai Yang, CNWRA, Ph.D., Technical Specialist

Third-Party Observers

Susan Lynch, State of Nevada

Don Shettel, State of Nevada

Engelbrecht von Tiesenhausen, Clark County, Nevada

Ram Murthy, DOE, Office of Quality Assurance

4.0 REVIEW OF THE AUDIT AND AUDITED ORGANIZATION

The BSC auditors conducted the audit in accordance with Line Procedure (LP)-18.6Q-BSC,

“QA Internal Audit Program,” and Administrative Procedure (AP)-16.1Q, “Condition Reporting

and Resolution.” The audit team used elements of the QARD, applicable BSC procedures,

LLNL procedures, and the two analysis model reports, to generate the audit checklist. The

NRC observers followed NRC Manual Chapter 2410 “Conduct of Observation Audits,” while

observing the audit. 
4.1 Scope of the Audit

The objective of the audit was to evaluate whether scientific investigations related to waste

package and drip shield degradation provided input into selected models that resulted in

technically sound and defensible model results. The models selected for review were

ANL–MD–EBS–000003, “General and Localized Corrosion of the Waste Package Outer

Barrier,” and ANL–MD–EBS–000033, “Engineered Barrier System: Physical and Chemical

Environment.” The audit included a review of inputs to these models back to the original data

collected and an evaluation of scientific approaches and methods. The audit scope also

examined the use and control of measuring and test equipment, implementation of QA

requirements, and the collection, documentation, control, and use of data. QA requirements

examined during this audit were contained in DOE’s QARD and applicable DOE and BSC

quality-affecting procedures. 

4.2 Conduct and Timing of the Audit

BSC provided its original audit plan to NRC on July 25, 2005. However, on August 3, 2005,

less than two weeks before initiation of the audit, BSC issued a revised audit scope that

reduced the areas to be reviewed during the audit. This reduction in audit scope brought into

question whether the audit objectives, stated as a broad assessment of data input and technical

products, could be fulfilled. As a minimum, the change in scope weakened the audit and

reduced the number of products that would be reviewed. Overall, NRC finds that the reduced

audit scope also limited the opportunities to identify issues with the data input and technical

products.

NRC also observed that preparations for the audit were delayed, in that audit checklists were

not complete and available until the start of the audit. This was inconsistent with past

experiences related to audits, conducted by DOE and BSC, in which the audit scope and audit

checklists were generally completed well in advance of the audit activity, allowing project

personnel and NRC staff to adequately prepare for the audit. 

NRC finds that reducing the scope of the audit at the last minute and not having the audit

checklists available until the beginning of the audit represented an audit weakness.

4.3 Audit Team Qualifications and Independence

The NRC observers reviewed the qualifications for the audit team members, including the

technical specialists assigned to the team, and determined they were qualified and independent

of the areas reviewed. Several senior-level QA personnel were included on the BSC audit

team. These individuals were cognizant of the DOE and BSC QA requirements and the various

procedural requirements related to the type of work being reviewed during this audit. The

technical specialists assigned to the audit were knowledgeable in the areas reviewed and asked

in-depth questions during interviews. However, BSC did not include either of the two technical

specialists in the LLNL portion of the audit, thereby reducing the effectiveness of the audit to

identify technical issues with the experiments being conducted at LLNL. This concern was

expressed to the BSC audit team during the audit by NRC. Not including the technical

specialists in the LLNL portion of the audit is identified as an audit weakness.

4.4 Examination of QA Elements
The following activities were audited by the BSC QA specialists on the audit team. 

4.4.1 QA Program

The BSC auditors reviewed planning-related documents to verify that sufficient detail was

included to support testing and validation and to allow a technically competent independent

individual to replicate the work. Technical work plans were reviewed to determine that qualified

personnel were assigned to perform the scientific investigation work. Software, measuring, and

test equipment and sample selection were reviewed for appropriateness. Qualification records,

for selected personnel responsible for performing, verifying, and managing scientific activities

associated with the audited models, were reviewed, to verify that these aforementioned

individuals’ education and experience were commensurate with the requirements established in

the job descriptions for the assigned positions. It was also confirmed that these individuals had

completed training as specified in the assigned training program descriptions. No NRC issues

were identified related to this audit topic.

4.4.2 Procurement Document Control

The BSC audit team reviewed a representative sample of scientific notebooks and associated

procurement documents to determine if the purchases made in support of scientific activities

were in compliance with quality-related procedures. Procurement document records for these

purchases were reviewed to verify that appropriate item or service descriptions were included

and quality, technical, and acceptance requirements were specified for items and services. No

NRC issues were identified related to this audit topic.

4.4.3 Implementing Documents

The BSC auditors verified that management was responsible for assuring work was conducted

in accordance with appropriate technical implementing procedures. Implementing documents

were reviewed and verified to address work modifications or changes, data collection, and

analysis, and verifications and traceability. No NRC issues were identified related to this audit

topic.

4.4.4 Document Control

The BSC auditors reviewed and verified that controlled documents were distributed through

electronic notification with links to the controlled document in the Control Document Information

System. Paper copies being maintained were verified to have the appropriate required red

stamp on the first page. 

The NRC observers noted that some of the analysis model reports reviewed during the audit

contained references to documents that were “canceled.” Specifically, the “Drip Shield and

Waste Package Outer Barrier,” analysis model report (ANL–EBS–MD–000001, Revision 1),

which had been cancelled, was referenced numerous times in the two analysis model reports

evaluated during the audit. This cancelled analysis model report was not in the current report

database. Subsequent to the identification of this issue, the NRC observers were advised that

referencing cancelled documents is considered an acceptable BSC practice. However, the

referencing of canceled documents could be problematic during the potential license application

review, and NRC has requested clarification from DOE regarding its plans to use cancelled

documents. This request has been submitted as Audit Observation Inquiry AOI-OAR-05-05-04. 

4.4.5 Control of Purchased Items and Services
The BSC auditors reviewed scientific notebooks to evaluate items and services that had been

purchased. The BSC auditors verified that acceptance reports were completed, that items and

services met purchasing requirements, deficiencies were resolved with suppliers, and reports

were transmitted to BSC procurement for inclusion with the project data package. No NRC

issues were identified related to this audit topic.

4.4.6 Control of Measuring and Test Equipment

The BSC auditors reviewed the scientific notebooks associated with the LLNL corrosion study

experiments, to identify the specific equipment used in these tests. This included a review of

records associated with calibrated equipment for service procurement, receipt inspection,

labeling, recall, calibration standards used, calibration method, and out-of-tolerance and

nonconformance report documentation. The review included examination of two previously

documented Condition Reports (CRs). Specifically, CRs 2247 and 5430 documented the use

of the Vaisala Model HMP-243 Temperature/Humidity Probes at temperatures in excess of their

calibrated range. The use of these probes was also in excess of the range for which the

manufacturer certified the accuracy of the indicated relative humidity. 

The information identified by the NRC observers during the review of the two CRs 2247 and

5430 brought into question the validity of the data collected, when the subject humidity probes

were used, during the corrosion experiments conducted between the years 2002 and 2005 at

LLNL. Based on the result of the review of the CRs and observation of BSC interviews with the

LLNL researchers and managers, the NRC observers questioned whether the corrosion data

could be considered technically sound and defensible. The NRC observers concluded that

LLNL had started at least two experiments in non-compliance with the QARD calibration

requirements and had conducted several years of data collection with the humidity probes not

calibrated and used outside the manufacturer’s documented range of acceptable use. 

At the end of the audit, BSC concluded that LLNL had effectively implement the QARD. The

NRC observers are concerned about this conclusion, based on the following issues related to

QARD Section 12.0, “Control of Measuring and Test Equipment,” and BSC

Procedure LP–12.1Q–BSC, “Control of Measuring and Test Equipment”: 

! Experiments Were Started Without Calibrated Instruments: Two experiments were

identified, during this audit, that had been started without the Vaisala humidity probes

calibrated for the high range of temperatures used in the experiments. Specifically,

QARD Section 12.2.1 states, in part, that ... “instruments shall be calibrated prior to use

against reference calibration standards traceable to nationally recognized standards.” 

Despite this QARD requirement, experiments related to Scientific Notebook

SN–LLNL–SCI–474–V2, “Mixed Salts,” and Scientific Notebook SN–LLNL–SCI–484–V1,

“Deliquescence of Mixed Salts, Measurements of Deliquescence Relative Humidity and

Determination of Boiling Point from Deliquescence Relative Humidity,” were started

without the Vaisala humidity probes being calibrated for the expected temperature

ranges of the experiments.

! Calibration Basis Was Not Documented: LLNL did not comply with QARD Section

12.2.1, concerning establishing a basis for calibration when nationally recognized

standards are not available. Specifically, QARD Section 12.2.1 states, in part, that “If no

nationally recognized standard exists, the basis for calibration shall be documented.”

Despite this QARD requirement, no basis for calibration was established by LLNL before
using the Vaisala humidity probes in two of the experiments reviewed during this audit. 

! LLNL Planned to Conduct an In-House Experiment to Calibrate Probes and Qualify Data

after the Experiments Were Completed: In response to the problem documented in

CRs 2247 and 5430, LLNL devised an in-house experiment to “calibrate” the Vaisala

humidity probes at the higher temperatures. This was required because the

manufacturer declined to certify that the probes were appropriate for use at the higher

temperatures or to provide calibration documentation, and it was determined that a

National Institute of Standards and Technology traceable standard did not exist. Once

the probes were “calibrated,” LLNL planned to declare the data collected during the SCI474 and SCI-484 experiments “qualified,” despite the fact that the probes had not been

calibrated before starting the experiments. No industry standard protocol, such as that

identified in ANSI/ISO/IEC 17025:2005(E), had been adopted as the basis for

performing the independent “calibration” of the probes. Additionally, no process for an

independent validation of the LLNL calibration results had been identified to ensure the

calibration experiments could withstand a rigorous scientific examination. NRC also

noted that no provisions are included in the QARD for starting an experiment with an

instrument not calibrated at the temperatures in which it will be used, and then

calibrating it after completion of the experiment and considering the experimental data

as qualified. 

! Scientific Notebook SCI–484 States Probes Were Calibrated: Scientific Notebook

SCI–484 documented that the Vaisala humidity probes were calibrated and provided the

calibration date for the probes in the scientific notebook. Scientific Notebook SCI-484

was initiated 6 months after the calibration problem was identified and documented in

CR 2247. However, no explanation nor notation was included in the scientific notebook

identifying the humidity probes as not being calibrated at the high range of temperatures

being used in the experiments. Although this information was provided to the BSC audit

team, it did not elevate this misrepresentation of the calibration status of the probes in

the scientific notebook to DOE and BSC managements for further investigation.

NRC determined that LLNL had not complied with the requirements of Section 12.0 of the

QARD related to the performance of corrosion rate experiments. Specifically, it was concluded

that LLNL had initiated these experiments: (i) without properly calibrated instruments; (ii) had

not established an adequate basis for calibration when nationally recognized standards were 

unavailable; and (iii) misrepresented the correct status of the calibration of the humidity probes

in the associated scientific notebook. Based on the issues identified above, NRC concluded

that LLNL had not satisfactorily and effectively implemented the QARD requirements related to

the control of measuring and test equipment. This conclusion disagreed with the conclusions

reached by the BSC audit team. Therefore, NRC is requesting an explanation as to why these

issues did not constitute an unsatisfactory implementation of the QARD at LLNL. This request

has been submitted as Audit Observation Inquiry AOI-OAR-05-05-05.

4.4.7 Handling, Storage, and Shipping

While at LLNL, the BSC auditors verified that samples were handled, stored, and shipped

adequately to prevent damage, deterioration, and loss. Controls, inspections, and tests for

special handling tools and equipment were conducted in accordance with documented

procedures. Specified time intervals were in place to ensure that tools and equipment were

adequately maintained. No NRC issues were identified related to this audit topic.
4.4.8 Corrective Action

The BSC auditors reviewed a number of CRs related to the LLNL experiments. Of the 14 CRs

issued, 12 were identified by the LLNL QA organization and only 2 were identified by the

researchers. Based on the NRC observer’s discussions with the LLNL researchers, the

threshold at which a CR was required was based on the significance of the issue and whether

the problem could be easily resolved. Specifically, if the problem were significant or could not

be resolved quickly, a CR was initiated. However, low-significance issues that were easily fixed

were typically not documented in the corrective action system. As noted by the NRC observers,

this approach to problem identification for quality-affecting activities does not comply with the

requirements of Procedure AP-16.1Q, “Condition Reporting and Resolution.” Specifically,

Procedure AP-16.1Q states that, “Conditions shall be entered into the Corrective Action

Program (CAP) system as soon as practical after identification.” Furthermore, Procedure AP16.1Q does not establish a threshold that provides for easily fixed and low-significance issues

to be excluded from the requirement to issue a CR. Allowing staff to make a determination as

to when an issue is entered into the corrective action program, as opposed to requiring all

issues to be entered into the system, is identified as a deficiency. 

Within the scope of the audit, two CRs were reviewed in detail by the NRC observers. These

were: (i) CR 2247, “Measuring and Test Equipment Used Outside the Range of Calibration,”

initiated February 19, 2004, and closed December 1, 2004; and (ii) CR 5430, “Data Acquired by

Vaisala Temperature/Humidity Probes Above 90 Degrees Celsius [194 °F] Is Indeterminate,”

initiated April 8, 2005. CR 5430 was still open at the time of this audit. Both CRs concerned

the use of the Vaisala probes to make relative humidity measurements outside their calibrated

temperature range. Specifically, the probes were calibrated at temperatures of 20 °C, 40 °C,

and 60 °C, for use in collecting relative humidity readings. The NRC observers noted that a

May 12, 2004, entry in the “Investigation Findings” section of CR 2247 documented that the

industry current standards do not support traceable calibrations above 70 °C, and that there is

no current standard that can be used to properly calibrate humidity probes at temperatures

above 70 °C. CR 2247 also identified that the probes had been used at temperatures up to

90 °C, and CR 5430 references test temperatures above 120 °C. Additionally, CR 5430

identified an action to construct a system for calibrating the probes up to 180 °C. The 180 °C

value was the upper temperature limit being used in the SCI-484 experiments, which was not 

mentioned in either of the CRs. The NRC observers further noted that CR 2247 was issued as

a Significance Level B. However, the subsequent problems, documented in CR 5430, that

were repetitive to the deficiencies identified in CR 2247, were characterized as a Level C. No

explanation for the disparity in CR significance levels was provided during the audit, and the

BSC audit team did not identify this issue as an item requiring follow-up. 

The specific concerns, identified by the NRC observers related to the corrective action program

aspects of these CRs are discussed below. 

! Corrective Actions for CR 2247 Not Implemented as Described: CR 2247 identified a

deficiency regarding the use of relative humidity probes outside their calibrated range. 

The remedial actions associated with this CR stated that ... “investigators will document

(make reference to) CR 2247 in open scientific notebooks identified as having used the

Vaisala temperature/humidity probes outside their range of calibration.” This entry in

CR 2247 was made three months before Scientific Notebook SCI–484 being opened. 

However, Scientific Notebook SCI–484 does not include a reference to CR 2247, even

though CR 2247 was still open when the experiments for Scientific Notebook SCI-484
were started. Therefore, once the initial review of the open scientific notebooks had

been completed as a remedial action for CR 2247, no provisions were established to

ensure that future scientific notebooks that used the Vaisala probes would reference CR

2247. 

! Out-of-Calibration Measuring and Test Equipment Were Not Controlled: The Vaisala

humidity probes identified in CR 2247 were not tagged, segregated nor otherwise

controlled to prevent use until they had been calibrated. This is a requirement of

Section 12.2.3.B of the QARD. No objective evidence was presented, during the audit

at LLNL, that the Vaisala probes had been tagged with a Measuring and Test

Equipment (M&TE) Limited Calibration sticker in accordance with the requirements of

Procedure LP–12.1Q–BSC, “Control of Measuring and Test Equipment.” The M&TE

Limited Calibration sticker identifies the calibration limitations of an instrument and could

have been a valuable part of the corrective action program to prevent use of the probes

in future high-temperature experiments.

! CR 5430 was Opened Without Recognizing That Additional Experiments Were

Underway with the Uncalibrated Probes: CR 5430 was opened April 8, 2005,

concerning the 2002 mixed-salt experimental results that were entered into the Data

Tracking Number (DTN) System as “qualified.” CR 5430 was a follow-up of issues

related to CR 2247. A review of the DTN information related to the mixed-salt data

found that the data were entered between November 2002 and February 2003 and

remained in the system as qualified until July 2005, a period of approximately 2 1⁄2 years.

CR 5430 discussed the problem with the calibration of the Vaisala probes and

referenced CR 2247. However, CR 5430 did not identify that experiments related to

Scientific Notebook SCI–484 were currently underway and had been underway for 8

months, also using the Vaisala probes outside their calibrated range. 

! LLNL Had Not Issued a CR for the Second Set of Experiments Using the Uncalibrated

Probes: The experiments related to Scientific Notebook SCI-484 had been initiated in

August 2004. As of the date of this audit in August 2005, no CR had been issued

related to the use of the uncalibrated humidity probes in the Scientific Notebook SCI-484

experiments. LLNL failure to recognize this repetitive non-conformance with the QARD

requirements related to instrument calibration and issue a CR is an example of the

ineffectiveness of the implementation of the QARD at LLNL, related to the corrective

action program. 

The NRC observers determined that LLNL had not complied with the requirements of

Procedure AP-16.1Q related to documenting deficiencies, preventing recurrence of the

problems, and capturing the second set of experiments using uncalibrated probes in the

corrective action program. Based on the issues identified above, NRC concluded that LLNL

had not satisfactorily and effectively implemented the QARD requirements related to corrective

actions. This conclusion disagreed with the conclusions reached by the BSC audit team. NRC

is requesting an explanation as to why these issues did not constitute an unsatisfactory

implementation of the QARD. This request has been submitted as Audit Observation Inquiry

AOI-OAR-05-05-05. 
4.4.9 QA Records

After task completion by project personnel, document packages are submitted to the records

processing center. The BSC audit team reviewed the availability of selected records that had

been submitted to the records process center through both the computerized system and, for

records not yet available in the computerized system, the ability to access the paper copies of

the records. The BSC auditors identified several minor deficiencies specifically related to the

records management process. These included instances of late record submittals, incorrect

sensor identification in an acceptance report, and a signature missing from the document

review record. 

During the review of various records related to ANL–MD–EBS–000003, “General and Localized

Corrosion of the Waste Package Outer Barrier,” Revision 2, the NRC observers noted that

scanning electron microscope images related to the corrosion-rate experiments were not

available in the technical data management system. Further review found that the documents

had not been submitted as part of the project records. These documents, identified as Binder

#11, were eventually located at LLNL. In the experiments discussed in these documents,

corrosion rates were determined by cleaning the samples of all corrosion, then examining the

sample under the scanning electron microscope, to verify all corrosion had been removed. If

corrosion were still present on the sample, the cleaning process was repeated. Once all

corrosion was removed, the sample was weighed and compared to the weight before cleaning,

to determine the weight loss. From these data, the corrosion rate could be determined. 

However, NRC observers examined the images in the documentation package and determined

that all of the scale had not been completely removed from the samples. This finding was

inconsistent with the information in ANL–MD–EBS–000003, which stated that all of the scale

had been removed from the samples. The NRC observers noted that the incomplete cleaning

of all the surface corrosion from the samples would impact the determination of the corrosion

rate values.

In summary, the failure to submit Binder #11 as part of the project records, the incorrect

statement in the associated analysis model report related to the final condition of the corrosion

samples, and the concern related to the impact of inadequate cleaning of the samples on the

conclusions reached on the corrosion rate values is identified as a deficiency. The NRC

observers identified these issues to the BSC audit team; however, no CR was issued. The

failure to issue a CR to document these conditions is identified as a deficiency.

4.4.10 Software, QARD Supplement I

The BSC auditors reviewed software used during data collection and determined that all the

software was off-the-shelf software that did not require independent verification and validation. 

No NRC issues were identified related to this audit topic.

4.4.11 Sample Control, QARD Supplement II

The BSC auditors examined the pertinent scientific notebooks and identified samples for

review. Inspection records were accessed though the technical data management system. 

Implementation of controls was reviewed during the onsite audit at LLNL. The BSC auditors

verified that samples were being controlled in accordance with the applicable procedures. 

Sample identification, location, transfer, storage life, segregation, traceability to the appropriate

logs, and other records were verified. No NRC issues were identified related to this audit topic.
4.4.12 Scientific Investigation, QARD Supplement III

During this audit, selected scientific notebooks were reviewed by the BSC audit team. The

notebooks were reviewed to determine completeness, level of detail, traceability, and the

documentation of compliance reviews. Four of the 14 scientific notebooks were found to have

problems. These problems included inadequate entries, traceability of information, incomplete

information, and lack of detail that would allow confirming the results without additional

information from the original investigator. The BSC audit team issued two Level B CRs related

to incomplete information in the scientific notebooks. No NRC issues were identified related to

this audit topic.

4.4.13 Control of the Electronic Management of Data, QARD Supplement V

The BSC audit team technical specialists evaluated the transparency of technical products

through document and data reviews and interviews conducted with the appropriate model

development staff. The BSC audit team also examined the inputs that were documented in the

technical products themselves and verified that all data were compiled in accordance with the

appropriate technical data management system database requirements. The BSC auditors

identified a discrepancy regarding the absence of a To-Be-Verified (TBV) number for data used

in an approved document. No NRC issues were identified related to this audit topic.
4.4.14 Monitored Geologic Repository, QARD Appendix C

The BSC auditors reviewed documentation associated with purchased items and services

where alternatives to QARD Section 7.0 were identified. Specifically, these included the

procurement of quality-affecting services from non-quality supplier laboratories. In these cases,

the BSC auditors verified the existence of the quality control sample plan and that the principal

investigator confirmed that the analytical results received have been evaluated to the

acceptance criteria established by the applicable quality control sample plan. No NRC issues

were identified related to this audit topic.

4.5 Examination of Technical Activities

The BSC audit team technical specialists reviewed several documents supporting the

engineered barrier system model, and conducted interviews with the technical report authors,

checkers and reviewers. During the review of the associated technical data, the NRC

observers identified several issues that were documented as audit observation inquiries. Based

on the significance of these issues, NRC requests that BSC provide additional information, that

can be used to evaluate the effectiveness of the BSC audit team, and to evaluate the

implementation of the QA program by the organization being audited.

4.5.1 ANL–EBS–MD–000033, “Engineered Barrier System: Physical and Chemical

Environmental Model,” Revision 4

The BSC technical specialist reviewed the engineered barrier system physical and chemical

environment model and the other documents that support this analysis model report, in

conjunction with the audit team QA specialists, who interviewed the documents’ originators,

reviewers, and team lead. 

4.5.1.1 MDL–NBS–HS–000001, “Drift-Scale Thermal-Hydrological-Chemical (THC)

Seepage Model,” Revision 4, dated February 2005

The BSC technical specialist reviewed various scientific notebooks and documents and

conducted numerous interviews with individuals involved with collecting and analyzing the data

for the Drift Scale THC Seepage Model. As a result of these review activities and interviews,

the BSC audit team identified the following technical discrepancies:

! The notebook that described the minerals used in the report (Section 6.2.2.2) did not

provide transparent and traceable information to support the selection of the minerals.

! The determination of the mineral reactive surface areas described in Appendix B, that

was used as input to the rate equations in Section 6.4.2, was not properly documented

and referenced.

! The documents that provided the basis for the selection of some thermodynamic data in

the model report (Section 4.1.4) were not referenced in the report, nor were they listed

in the references.

! The basis for the selection of the uncertainty used in the analysis for the

mineral dissolution and precipitation rate was not properly documented, nor was

it referenced.
! The description for the selection of the waters (PERM–2 and PERM–3) was

not transparent. 

As a result of the technical discrepancies found in this report, the BSC technical specialist

determined that this model was unsatisfactory. BSC issued a Level B CR. 

The NRC observers agree with the conclusions of the BSC technical specialist . In addition, the

NRC observer questioned the technical basis for the assumption that no dripping will take place

when the temperature of the drift wall is above 100 °C (212 °F). The Engineered Barrier

System Physical and Chemical Environment Model Report assumes a 100 °C (212 °F)

threshold for the onset of seepage. The statement, “Seepage waters do not enter the drifts

until the host rock temperatures fall below 100 °C (212 °F),” is not supported by sufficient

justification. BSC is being asked to provide the basis for why it did not consider the salt effect

on the boiling point. This request will be tracked as audit observation inquiry AOI-OAR-05-05-

01.

4.5.1.2 ANL–EBS–MD–000074, “Analysis of Dust Deliquescence for Features,

Events, and Processes (FEP) Screening,” Revision 0, dated June 2005

The technical basis for the selection of the salt assemblage and the adequacy of the database

in ANL-EBS-MD-000074 were reviewed. Based on this review, the BSC team identified a

technical discrepancy related to adequate justification for the selection of the log K value for

sepiolite in the validation of the model. 

The NRC observers also questioned the validated relative humidity range for the model and

why the deliquescence relative-humidity data were presented in the report below the modelvalidated range of 40 percent. The EQ3/6 model for deliquescence relative humidity (RHd) was

validated for a relative humidity range from 100 to 40 percent. This is documented in the notes

for Table 7-8 of ANL–EBS–MD–000045, “In-Drift Precipitates/Salts Model,” Revision 2. 

However, ANL–EBS–MD–000074 presented results, in Figure 6.2-5, that have deliquescence

relative humidity values much lower than 40 percent. Similarly, the equilibrium relative humidity

data calculated by the EQ3/6 model for solutions containing mixed salts were well below 40

percent, as presented in Figures 6.13-2 and 6.13-13 of ANL–EBS–MD–000033, “Physical and

Chemical Environment Model,” Revision 4. BSC is requested to provide the basis for using

model results that are outside of the model validation range. This request will be tracked as

audit observation inquiry AOI-OAR-05-05-02.

4.5.1.3 ANL–DSD–MD– 00001, “Aqueous Corrosion Rates for Waste Package

Materials,” Revision 1, dated October 2004

Based on discussion between the BSC technical specialist and the researchers, BSC identified

a technical discrepancy concerning the adequacy and transparency of documentation in this

report. The discrepancy related to the criteria the investigator followed to select or exclude the

corrosion rate data compiled in the report.

4.5.1.4 ANL–EBS–MD–000045, “In-Drift Precipitates/Salts Model,” Revision 2,

dated October 2004

The BSC technical specialist conducted interviews regarding the accuracy in uncertainty
represented in the technical basis for this model. The uncertainty range of the output pH from

the model had been developed by comparing the model results with four experimental data

points. The BSC technical specialist determined that this was not adequate. This

determination was documented on Condition Report CR6487. 

4.5.1.5 ANL–EBS–MD–000033, “Engineered Barrier System: Physical and

Chemical Environment,” Revision 4, dated July 2005

The BSC technical specialist conducted interviews regarding the accuracy, and appropriateness

of the Pitzer database, and the completeness and accuracy of the calculations for the seepage

evaporation in the model. As previously noted, the NRC observer questioned why the relative

humidity data was below 40 percent, which is the lower limit that the Pitzer model was validated,

were presented in the report. (Reference AOI OAR-05-05-02)

4.5.2 ANL–EBS–MD–000003, “General Corrosion and Localized Corrosion of Waste

Package Outer Barrier,” Revision 2

The BSC technical specialists reviewed the sections of this model that included the dry

oxidation model parameters; general corrosion temperature dependence modeling; localized

corrosion model logic and data interpretation; experimental methods for corrosion tests;

microbiological-induced corrosion modeling; statistical methodologies for corrosion-rate

distributions; and associated model data from supplementary technical documents

ANL–EBS–MD–000003, and ANL–WIS–MD–000023D, “Analysis of Waste Package Outer

Barrier Localized Corrosion.” As a result of the NRC observers’ evaluation of the technical

information related to this model, the following observations have been made. 

4.5.2.1 Corrosion of Titanium Grade 24 Based on Grade 7 Data

The BSC technical specialist conducted interviews regarding the selection of corrosion-rate

measurements in ANL–DSD–MD–000001, “Aqueous Corrosion Rates for Waste Package

Materials,” Revision 1. The NRC observers reviewed this document as well and identified the

lack of adequate justification for using the general corrosion rates for the single phase (")

Titanium Grade 7 alloy in the modeling of the dual phase ("/$) Titanium Grade 24 alloy in

representative repository environments (see Section 6.5, page 6-39 of

ANL–DSD–MD–000001). No technical justification was provided for the basis that the

measured corrosion rates of Titanium Grade 7 can be extrapolated to the Titanium Grade 7/24

weldments in the as-welded and stress-relieved conditions. This issue will be followed by NRC

and will be discussed at future DOE and NRC technical meetings.
Specifically, the following information is needed to make a determination of the adequacy of the

information related to the corrosion rates.

! A technical basis for modeling the general corrosion rates of the dual-phase ("/$)

Titanium Grade 24 alloy in a representative repository environment using the general

corrosion rates from the single-phase (") Titanium Grade 7.

! A technical basis for the general corrosion rates for Titanium Grade 24 base material

and Titanium Grade 7/24 weldments in the as-welded and stress-relieved conditions, as

would be expected on an as-fabricated drip shield in a representative repository

environment. 

The lack of an adequate justification for the Titanium Grade 7/24 weldments and Titanium

Grade 24 general corrosion rates under representative repository conditions is identified as a

significant weakness related to this analysis model report and an area of technical concern for

the potential license application. 

4.5.2.2 Corrosion Scaling Factor for the Microbiologically Induced Corrosion

Model 

The BSC technical specialist conducted interviews regarding the adequacy of the models in

ANL–EBS–MD–000003, “General Corrosion and Localized Corrosion of Waste Package Outer

Barrier.” Based on the review of this information, the NRC observers identified the need for

additional justification for modeling microbiologically influenced corrosion, using a scaling factor

to the generalized corrosion model. The scanning electron micrographs of the corrosion under

biotic conditions in Figure 6-67, of ANL–EBS–MD–000003, appear significantly different from

typical general corrosion. Specifically, the NRC observer noted that micropores were found, on

the specimens, that were tested under the biotic condition. This may be an indication of

localized corrosion, as opposed to general corrosion. Although the general corrosion scaling

factor was intended to represent a conservative approach, the exclusion of localized

microbiologically influenced corrosion effects without a documented justification is identified as

a weakness in the modeling used for this analysis model report.

4.5.2.3 Inconsistencies for Overall Localized Corrosion Modeling

The BSC technical specialist conducted interviews regarding the technical adequacy of the

models in ANL–EBS–MD–000003, “General Corrosion and Localized Corrosion of Waste

Package Outer Barrier.” The NRC observers identified inconsistencies between the localized

corrosion model criteria found on page 8-6 of ANL–EBS–MD–000003, “General Corrosion and

Localized Corrosion of Waste Package Outer Barrier,” Revision 2 as compared to the model

criteria found in ANL–EBS–MD–000074, “Analysis of Dust Deliquescence for Features, Events,

and Processes Screening,” Revision 0. For example, ANL–EBS–MD–000003, states on pages

1-3, 6-97, and 8-6, “If the exposure temperature exceeds 160 °C [320 °F] and a water film is

present on the waste package surface, then localized corrosion initiates”; however,

ANL–EBS–MD–000074 considers that localized corrosion will not initiate when a water film is

formed by deliquescence at elevated temperatures (page 7-1), even though it has been shown

experimentally that some salt assemblage may deliquesce at temperatures over 190 °C

[374 °F] (page 7-2). BSC is requested to provide an explanation for these inconsistencies. 

This request will be tracked as audit observation inquiry AOI-OAR-05-05-03.
4.6 Potential Audit Findings

The BSC audit team identified the following potential CRs:

Three conditions adverse to quality were identified as Level B CRs:

1. Inadequate entries in Scientific Notebooks SN–LLNL–SCI–484–V1 (LP–SIII.11Q–BSC);

2. Inadequate documentation and traceability of work on MDL–NBS–HS–000001 “DriftScale THC Seepage Model,” Revision 4 (LP–SIII.10Q–BSC); and

3. Inadequate scientific notebooks supporting scientific investigations for

MDL–NBS–HS–000001.

Eleven minor conditions adverse to quality were identified as Level C CRs:

1. Licensing Support Network-relevant records not submitted to the records processing

center within 2 weeks (LP–IM–026–BSC);

2. Errors in calibration documentation (AP 17.1Q and LP–12.1Q–BSC);

3. Incorrect identification of a sensor in an acceptance report (AP–17.1Q);

4. Out-of-calibration report not changed to document out-of-calibration condition

(LP–12.1Q-BSC);

5. QA records not submitted to the records process center within 60 days (AP–17.1Q);

6. Signature missing from document review record (LP–2.14Q–BSC);

7. Documentation errors in ANL–EBS–MD–000005 (LP–SIII.10Q);

8. No to-be-verified (TBV) number for data used in approved document (LP–3.15Q–BSC);

9. Need for additional confidence in development of uncertainty range for pH output from

ANL–EBS–MD–000045, “In-Drift Precipitates/Salt Model,” Revision 2,

(LP–SIII.10Q–BSC);

10. Need to document the process for selecting aqueous corrosion-rate data in

ANL–DSD–MD–000001, “Aqueous Corrosion Rates for Waste Package Materials,”

Revision 2, (LP–SIII.10Q–BSC); and

11. Need to examine the effects of uncertainties in the sepiolite log K on the output

concentrations of magnesium in ANL–EBS–MD–000074, “Analysis of Dust

Deliquescence for FEP Screening,” Revision 0 (LP–SIII.10Q–BSC).

Two Recommendations were identified (Level D CRs):

1. Improve the description of critical potential measurement in ANL–EBS–MD–000005,

“General Corrosion and Localized Corrosion of the Waste Package Outer Barrier,”
Revision 2; and 

2. Clarify the decision tree for localized corrosion in ANL–EBS–MD–000005.

5.0 NRC STAFF FINDINGS

5.1 NRC OBSERVATION SUMMARY

Although most of the audit observation results were found to be acceptable, several areas were

identified as deficiencies or weaknesses. Deficiencies are a departure from a specific

requirement such as an applicable code, standard, guide, the QARD, or acceptable industry or

scientific practice. Weaknesses are activities which, though not a requirement, are less than

optimum for the audit or for the organization being audited.

The following weaknesses and deficiencies are identified in the body of the report and are

summarized below:

! Less than 2 weeks before initiation of the audit, BSC issued a revised audit scope,

which reduced the areas to be reviewed during the audit. The change in scope reduced

the number of products to be reviewed and reduced the opportunity to identify issues

with the data input and technical products. NRC also observed that preparations for the

audit were delayed in that audit checklists were not complete and available until just

before the audit. Reducing the scope of the audit at the last minute and not having the

audit checklists available until the beginning of the audit are identified as an audit

weakness. (Section 4.2)

! The technical specialists were not included in the audit activities at LLNL, thereby

reducing the effectiveness of the review of the experiments conducted at LLNL. This

was identified as an audit weakness. (Section 4.3)

! LLNL initiated experiments to study the deliquescence effect on the corrosion of the

outer surface of the waste package, using Vaisala humidity probes that were not

vendor-qualified nor calibrated for the upper temperature limits of the experiment. This

is in non-compliance with the QARD, which requires that test instruments be calibrated

before use. This deficiency is an example of ineffective implementation of the QARD, at

LLNL, related to the control of measuring and test equipment. (Section 4.4.6)

! LLNL planned to conduct experiments to calibrate the Vaisala humidity probes at the

high temperatures used during the corrosion experiments. However, no nationally

recognized standard existed, nor was the manufacturer able to provide a standard

process for performing a calibration at high temperatures. Also LLNL had not adopted a

nationally recognized protocol for conducting the calibrations. Not 

identifying these concerns to BSC and DOE management as potentially significant 

issues related to the defensibility of the LLNL experiments was identified as a 

deficiency. (Section 4.4.6)

! Scientific Notebook SCI-484 identified the Vaisala humidity probes as calibrated and

documented the calibration date for the probes. However, appropriate documentation
was not included, in the scientific notebook, identifying the limitation on the use of the

humidity probes at high temperatures. Failure to identify this potential misrepresentation

of the information in the scientific notebook to management for further investigation is

identified as a deficiency. (Section 4.4.6)

! Based on discussions with LLNL researchers, there was an understanding that

problems that could be easily rectified or that were minor in nature did not require entry

into the corrective action program. This practice is inconsistent with DOE’s procedures

concerning corrective action reporting, in that a threshold is not established for

determining whether a problem should be captured in the corrective action system. This

was identified as a deficiency. (Section 4.4.8) 

! LLNL had issued two CRs identifying problems with the use of the Vaisala humidity

probes. However, the corrective actions associated with these CRs were ineffective in

establishing adequate controls on the humidity probes to prevent their use in a second

set of experiments. This deficiency was identified as an example of ineffective

implementation of the QARD, at LLNL, related to the corrective action program. (Section

4.4.8)

! The second set of experiments using the Vaisala probes was started at LLNL, 1 year

before this audit as documented in Scientific Notebook SCI-484; yet, no CR had been

issued capturing this repetitive non-conformance with the QARD requirements related to

instrument calibration. LLNL has not established adequate provisions to prevent

recurrence of the problem. This deficiency was identified as an example of ineffective

implementation of the QARD, at LLNL, related to the corrective action program. (Section

4.4.8)

! A review of the list of documents submitted to the records processing center for the

“General and Localized Corrosion of the Waste Outer Barrier Analysis Model Report,”

determined that not all documents had been submitted. The missing documents related

to the corrosion-rate experiments. The documents were later located at LLNL. The

documents included images of the samples, used in the experiments that were suppose

to be cleaned of all corrosion; yet, the images appeared to still have corrosion present:

(i) the incomplete submittal of all documents to the records-processing center; (ii) the

lack of an accurate statement in the associated analysis model report concerning the

cleanliness of the samples; and (iii) the potential impact of the samples not being clean,

on the conclusions reached on the corrosion rate values are identified as deficiencies. 

NRC identified these issues to the BSC audit team. However, no CR was issued. Not

issuing a CR on these problems is also identified as a deficiency. (Section 4.4.9) 

! A review of ANL–DSD–MD–000001, “Aqueous Corrosion Rates for Waste Package

Materials,” Revision 1, identified a lack of adequate justification for using the general

corrosion rates for the single-phase (") Titanium Grade 7 alloy in the modeling of the

dual-phase ("/$) Titanium Grade 24 alloy in representative repository environments. No

technical justification was provided for the basis that the measured corrosion rates of

Titanium Grade 7 can be extrapolated to the Titanium Grade 7/24 weldments in the aswelded and stress-relieved conditions. The lack of an adequate justification for the

Titanium Grade 7/24 weldments and Titanium Grade 24 general corrosion rates under

representative repository conditions is identified as a significant weakness related to this

analysis model report and an area of technical concern for the potential license

application issue. (Section 4.5.2) 
! A review of ANL–EBS–MD–000003, “General Corrosion and Localized Corrosion of

Waste Package Outer Barrier,” identified the need for additional justification for

modeling microbiologically influenced corrosion, using a scaling factor to the generalized

corrosion model. The scanning electron micrographs of the corrosion under biotic

conditions in Figure 6-67, on page 6-106 of ANL–EBS–MD–000003, appear significantly

different from typical general corrosion. Specifically, the NRC observer noted that

micropores were found on the specimens that were tested under the biotic condition.

This may be an indication of localized corrosion, as opposed to general corrosion. 

Although the general corrosion scaling factor was intended to represent a conservative

approach, the exclusion of localized microbiologically influenced corrosion effects

without a documented justification is identified as a weakness in the modeling used for

this analysis model report. (Section 4.5.2)

5.2 NRC AUDIT OBSERVER INQUIRY 

In addition to the findings noted in Section 5.1 above, additional areas of technical product

weaknesses were noted. Seven audit observation inquiries (AOI) were initiated and submitted

to BSC for response. 

1. AOI-OAR-05-05-01: “Drift-Scale THC Seepage Analysis Model Report,” (Sections

4.5.1.1)

2. AOI-OAR-05-05-02: “Analysis of Dust Deliquescence for Features, Events and

Processes (FEP) Screening,” (Section 4.5.1.2)

3. AOI-OAR-05-05-03: “Inconsistencies for Overall Localized Corrosion Modeling,”

(Section 4.5.2.3)

4. AOI-OAR-05-05-04: “Referencing of Cancelled Documents,” (Section 4.4.4)

5. AOI-OAR-05-05-05: “Use of the Vaisala Humidity Probes at Temperatures Outside

their Calibrated Range,” (Sections 4.4.6 and 4.4.8)

.