### APPENDIX B

REVIEW OF WIPP PERFORMANCE ASSESSMENT

CO MPUTERIZED DATABASE (PAPD B ) MIGRATIONi 

### TABLE OF CONTENTS

Executive Summary

................................ ................................ ................................ ........................ ii

1.0 Introduction

................................ ................................ ................................ .......................... 1

2.0 General Approach

................................ ................................ ................................ ................ 3

3.0 Review of the PAPDB Migration

................................ ................................ ........................ 4

3.1 Rationale for Database Migration and Update

................................ ......................... 4

3.2 Migration Objectives

................................ ................................ ............................... 5

3.3 Approach

................................ ................................ ................................ .................. 5

3.4 Software

................................ ................................ ................................ ................... 5

3.5 Acceptance Criteria

................................ ................................ ................................ .. 6

3.6 Testing Methodology

................................ ................................ ............................... 6

3.6.1 Migration from SQL to MySQL Server

................................ ....................... 6

3.6.2 Code Functionality

................................ ................................ ....................... 6

3.6.3 T est Cases

................................ ................................ ................................ .... 8

3.6.4 Execution of Tests

................................ ................................ ........................ 9

3.7 Test Results

................................ ................................ ................................ .............. 9

4.0 Su mmary and Conclusions

................................ ................................ ................................ . 11

5.0 References

................................ ................................ ................................ .......................... 12ii 

### EXECUTIVE SUMMARY

The ability of the DOE WIPP facility to continue to meet the certification requirements of the

E PA is demonstrated in part through the use of a series of performance assessment (PA) computer codes. DOE must demonstrate on an ongoing basis that PA computer software is in compliance with regulations outlined in Section § 194.23

– Models and Computer Cod es and in ASME NQA

- 2a

- 1990 addenda, part 2.7

. Since the Agency’s certification of the DOE WIPP Compliance Certification Application (CCA), DOE has added computer hardware and upgraded the computer software

. In order to maintain compliance with Sections § 19 4.22, § 194.23 and ASME NQA

- 2a

- 1990 addenda, part 2.7 , DOE is required to conduct testing on the computer codes to ensure that they still function properly on new hardware and software.

This appendix describes the results of the U.S. Environmental Protecti on Agency’s (EPA or the Agency) review of Performance Assessment Parameter Database (PAPD B ) migration , design

modifications and testing activities performed by Sandia National Laboratories (SNL) in support of their ongoing Performance Assessment ( PA ) activ ities at the Waste Isolation Pilot Plant (WIPP).

In 201 1 , SNL migrated the PAPDB ( Version 1.0) used in the WIPP compliance calculations from the Microsoft SQL Server on a Windows Server to the PAPDB ( Version 2.00) on the MySQL

Server on Solaris. The purpo se of the migration is to facilitate access to the parameter data from the PA codes on Solaris, Linux and VMS. In addition, the design of the database was also modified to improve its ease of use.

SNL defines a parameter as any value or distribution of va lues used directly or indirectly as input

to a PA . The PAPDB contains the parameters used in WIPP PAs, the names of documents related to those parameters, the names of the codes that use the parameters, and a log of the parameters that were exported to the various codes during analyses. In the PAPD B a parameter may be represented by a single value or assigned a distribution which represents epistemic uncertainty. The values of parameters can change through time as new information becomes available . T he

data base contains the different versions of the parameter (s) which are identified by a version

number

, by the analysis where the parameter was first defined

, and by the effective date for its

first use. Specific versions of a parameter can be associated with o ne or more analyses. The purpose of the database is to securely store the parameter data used in the PA analyses, to allow access to that data directly from the applications that need those data, and to log the usage of the data. In addition, the PAPDB is designed to access the data from a user interface to facilitate querying the database to document the parameter values used in the various analyses

The primary objectives of the Agency’s review of the PAPD B database migration to the new platform are to as sess the transcription accuracy, documentation, and traceability of changes to the parameter values , distributions and metadata

. The PAPDB parameter database and associated metadata are maintained in Carlsbad, New Mexico, by SNL. Hard copies of documentati on supporting the parameters and their values are maintained in SNL’s WIPP Records Center in Carlsbad. The supporting documents are tracked by SNL’s Electronic Record Management System (ERMS), through which a unique identifying number is assigned to each d ocument.iii

The Agency has reviewed the PAPD B life

- cycle documentation (e.g., Requirements Document, Design Document, Verification and Validation Plan) t o determine whether the PAPD B migration to a new platform meet s the requirements specified in § 194.22

, § 194.23 and ASME NQA

- 2a

1990 addenda, part 2.7. In addition to the life

- cycle documentation , the Agency reviewed several migration reports to further evaluate the migration testing procedures and results (Kirchner and Long 2012 ; Kirchner 2011).

Based upon these reviews, the Agency found that the PA P DB verifica tion tests adequately test all the functional requirements necessary for PAPDB Version 2.00 to effectively su pport PA , and

that PAPDB Version 2.00 has successfully passed the tests

. F urthermore, the Ag ency determined

that SNL has developed documentation that is consistent with life

- cycle and testing requirements

outlined in § 194.22

, § 194.23 and ASME NQA

- 2a

- 1990 addenda, part 2.7

. The Agency therefore

concludes that PAPDB Version 2.00 is in compliance wi th §194.22, §194.23 and ASME NQA

2a

- 1990 addenda, part 2.7 and that PAPDB Version 2.00 is approved for WIPP PA use in PA

calculations

.1 

### 1.0 INTRODUCTION

This appendix describes the results of the U.S. Environmental Protection Agency’s (EPA or the Agency) revi ew of Performance Assessment Parameter Database ( PAPDB ) migration, design modifications and testing activities performed by Sandia National Laboratories (SNL) in support of their ongoing Performance Assessment (PA) activities at the Waste Isolation Pilot P lant (WIPP).

In 2011, SNL developed new database software, called PAPDB Version 2.00, to replace the previous PAPDB Version 1.00, and migrated the parameter values and associated metadata from

Version 1.00 to Version 2.00. The database structure was modif ied to make it easier to use and to facilitate parameter data access from the PA codes on Solaris, Linux, and VMS platforms. The

new database resi des on a MySQL Server and is physically located on an Intel

- based server running Solaris 11 (Kirchner and Long 2012). The database migration process is more fully described by Kirchner (2011).

A primary improvement in the new database is the ability to accurately retrieve parameter values used in earlier PAs. PAPDB Version 2.00 is designed to default to the param eter value most recently entered into the database, and to automatically log in the analysis that used the parameter value. However, a previous value can also be returned by specifying an earlier analysis. A similar functionality was provided in PAPDB Vers ion 1.00, but did not work proper ly (Kirchner 2011)

. In addition , distributed parameters are now assigned a default value when the distribution is not to be sampled. For example, when assigning three

- dimensional, isotropic permeability values to a

material , only the value of PRMX_LOG is randomly sampled using the PRELHS code. The constant default values for PRMY_LOG and PRMZ_LOG are taken from the database as placeholders and are subsequently reassigned the same value as PRMX_LOG in the BRAGFLO

ALGEBRA file

.

The methodology for retrieving parameter values from the database remains essentially unchanged. The MATSET code retrieves the default value for every parameter in the analysis. Parameter values that are to be sampled are identified in the input files to the PRELHS code, which supplies that information to the LHS code. The LHS code then retrieves 100 sample values per replicate from the database and supplies them to the post LHS code. If a parameter has been retrieved by LHS, the POSTLHS code replaces t he default parameter value retrieved by MATSET with the parameter value retrieved by LHS. In a subsequent step, the remaining default values retrieved by MATSET for distributed parameters are replaced by the desired value in the ALGEBRA1 file, as described in the example above. For constant parameters, the default value is the constant value and remains unchanged. Although the default values for distributed parameters are typically the means or medians of the distributions, any value can be used as the

defa ult value.

The PAPDB Version 2.00 database tables are constructed to store the data residing in PAPDB Version 1.00. An Access database, ParamTransfer.mdb, was used to transfer the data from

PAPDB Version 1.00 to PAPDB Version 2.00. Most of the transfers were done using queries to extract the data from the PAPDB Version 1.00 and append it to tables in the PAPDB Version2

2.00. The transfers were checked for completeness using a series of queries and, for the parameter value data, two VBA (Visual Basic for Ap plications) routines. The queries were designed to compare the PAPDB Version 2.00 values with the corresponding values in PAPDB Version 1.00 and to show only records in the PAPDB Version 1.00 that were not matched in the PAPDB Version 2.00. Thus, the desir ed outcome was an empty data set.

Verification of the transfer process involved a transfer of the data followed by the deletion of selected records from the various tables and execution of the various queries to show that the missing records were detecte d. Following the verification of the procedures, the data were transferred once again and the same queries run to verify that no records were dropped during the transfer. It was also necessary to generate a report that showed the records whose value string could not be matched due to differences in formatting. The records shown in that report were compared and verified to be equivalent.

In addition to the records transferred from PAPDB Version 1.00, two sets of records were developed that were missing from the original database. The design of the PAPDB Version 1.00 did not allow for multiple analyses being run under a single AP, as was done for AP

- 132. It appears to have kept only the records from the most recent retrieval of parameters. Furthermore,

the re trieval of parameters by MATSET for use in NUTS seems to never have been recorded. To capture these usages of parameters, a series of queries were run to insert the missing records.

The data were transferred following verification of the transfer procedur es. SNL and the Agency verified an accurate data transfer by reviewing the comparison reports, as documented by Kirchner and Long (2012). Problems found by SNL in initial data transfers were corrected and in the final transfer, all of the comparison tests were successful except one, where the data did not exactly match due to formatting differences. Those test results were reviewed manually and SNL

concluded that the values of the two data sets were found to be equivalent

. The Agency agrees

with this assess ment.

The remaining sections present the approach that the Agency followed to determine whether

PAPDB Version 2.00 is in compliance with § 194.22, § 194.23 and ASME NQA

- 2a

- 1990 addenda, part 2.7.3 

### 2.0 GENERAL A PPROACH

To demonstrate that computer software is in compliance with disposal regulations outlined in § 194 .22, SNL established a life

- cycle management process for software used to support their PA

activities

. Their qualification approach for the software follows the life

- cycle phases outlined in ASME NQA

- 2a

- 1990 addenda, part 2.7, as follows:

Planning Requirements Design Implementation

Validation

Installation and Checkout

Maintenance

Retirement

SNL implements the l ife

- cycle phases using an iterative or sequential approach and each phase is supported wit h relevant documentation (e.g. , Design Document, Verification Plan, and Implementation Document)

.

The Agency’s review of the adequacy of SNL ’s implementation and testing of the database migration was initiated in October 2012 with an on

- site review of the PAPDB and its PA code

interfaces. SNL maintains the parameter database and associated metadata in Carlsbad, New Mexico. Hard copies of documentation supporting the parameters and their values are retained in
SNL’s WIPP Records Center in Carlsbad

. SNL trac ks t he supporting documents by an Electronic Record Management System (ERMS) in which a unique identifying number is assigned to each

document.

The primary objectives of the Agency’s review of the database migration to the new platform

were to assess the adequacy of the documentation, traceability of the modifications and reliability

of the testing

. The scope of the review was primarily focused upon reviewing the following

documentation for PAPDB (Version 2) :

Analysis Plan (Kirchner 2011 ) Software Quality Assurance Plan ( Kirchner 2011a) Requirements Document ( Kirchner 2011b )

Verification Validation Plan (Kirchner 2011 c ) Design Document (Kirchner 201 2 )

Validation Document (Kirchner 201 2a )

In stallation and Checkout (Kirchner 201 2b ) Migration Report (Kirchner and Long 2012 )

T he transcription accuracy of the migration and t he traceability of changes to the parameter

values and metadata that were made for the PCS

- 2012 PA were also evaluated (Appendix A).4 

### 3.0 R EVIEW O F THE PAPDB MIGRATION

3.1 RATIONALE FOR DATABASE MI GRATION AND UPDATE

In 2011, the Parameter Database (PAPDB Version 1.0) was revised to Version 2.0 and migrated from the Microsoft SQL on a Windows Server to the MySQL software running Solaris 11. Prior to being migrated to MySQL, the parameter database ( P APDB ) was physically located on an

Intel

- based server also running Solaris 11, tgw.sandia.gov

. SNL migrated the database to facilitate access to the parameter data from the PA codes located on Solaris, Linux and VMS.

The primary function of PAPDB is to s upply parameter values for use in PA and PAPDB contains data values, associated codes, references for relevant documents, usage and additional information documenting the parameters used in PA. These data are exported to the other PA

codes via routines in the SDBREAD

_ LIB

. In PAPDB Version 1.00 the numeric data was encoded for a parameter, such as the mean, median and standard deviation, into an XML

formatted string. This was done to facilitate the display of the data via a web browser, which was the origina l goal for the database. SNL , however, notes that this design complicates the interface to the PA codes since the desired values must be unpacked from the XML string and converted

from text to numeric data.

To alleviate the complicated design, SNL redesi gned PAPDB Version 1.00 into Version 2.00 which has no stored procedures or queries

. In Version 2.00

, the business rules that had been implemented as stored procedures in Version 1.00 have either been eliminated through the redesign of the database or ha ve been implemented in the revision of SDBREAD

_ LIB. For example, the logging of parameter export by analysis and code is now the responsibility of

SDBREAD

_ LIB rather than implemented as a stored procedure. Thus, the PAPDB Version 2.00 provides storage for th e data but has no inherent functionality.

The specifications for the PAPDB Version 1.00 state that the parameter values obtained via SDBREAD_LIB should default to the most recent version unless an earlier version is specified (PAPDB Version 1.00 Requirem ents Document). SDBREAD _LIB contains the routines that are the interface between the performance assessment (PA) codes and the database. However, SNL

notes that this functionality appears to be flawed since the ability to specify earlier versions does

not seem to work properly (Kir c hner 2011) . Speci al analyses run by SNL to support PA over the last few years often require new parameter values, with subsequent return to the parameters of

the last PA baseline calculation. Thus , the typical approach that SNL h as followed has been to 1) enter the new values of some parameters for the PA, 2) run the PA, and 3) restore the values from the last baseline calculation by re

- entering them. This process is unproductive because it requires

a number of changes to the data base, adds redundant data to the database, and increases the chance of making an error when restoring b aseline values. Furthermore, SNL describes a problem that arises in the ‘ AnalysisID ’ field of the PAPDB (Version 1)( Kirchner 2011 )

. T his

field reflects t he analysis when parameters are first defined, but in some cases those parameters were not used in that analysis, e.g. the lower limit of the distribution assigned to

BOREHOLE:TAUFAIL was redefined in AP

- 13 1 ( Kirkes and Herrick 2006 ) and entered into the

d atabase but the PA for AP

- 131 did not use those new values. In this case, the code that used5

BOREHOLE:TAUFAIL, PRELHS, was not run and instead the LHS sample d values from a previous analysis were used in the AP

- 131 PA. As noted by SNL, n o record was entere d into the

‘ AnalysisRetrievalHistory ’ and thus the PAPDB Version 1 database c ould not be used to determine which version of the parameter was really used in the PA.

3.2 MIGRATION OBJECTIVES

SNL designed the MySQL version of the PAPDB (PAPBD Version 2.00) to store numeric values associated with parameters in a numeric field that allow the codes to specify a specific version of a parameter value to be returned by specifying an analysis.

To allow a determination of which version of the parameter was used in t he PA SNL added a

new table to map from an analysis name to a version for each parameter used in an analysis. In addition, parameters assigned distributions are assigned a ‘ value ’ property in the PAPDB Version

2.00 that are used as the default value when t he distribution is not to be sampled. In PAPDB

Version 1 .00 the median is used as the default value for most distributions and the mode for triangular distributions. SNL notes that t his has the consequence that the value assigned to the

median has not alwa ys been the true median of the distribution. SNL also indicates that t his

situation has no impact on the distribution sampled by LHS because for the majority of the distributions, excluding the triangular, cumulative and Student’s distributions, it is only the maximum and minimum values that are used to specify the distributions. SNL added the default

values in the PAPDB Version 2.00 to remedy this issue.

3.3 APPROACH

The PAPDB Version 2.00 database tables are constructed to store the data residing in the

PA PDB Version 1.00. SNL, however, simplified the structure of the database by eliminating the

many ‘ index ’ tables, such as the table dbo Material which lists the material name along with a

numeric index associated with the material name. The number of tables was reduced from 41 to

14 for inclusion in PAPDB Version 2

. A number of the tables in PAPDB Version 1 were meant for functions that were never implemented

. Other tables stored values that were always the same, or which were never used in the parameters vi ewer or to retrieve values for the codes. SNL did

not transfer t hese tables to PAPDB Version 2 . In some cases , SNL combined two or more tables

into one.

3.4 SOFTWARE

SNL used a n Access database, ParamTransfer.mdb, to transfer the data from PAPD B Version 1 to
PAPD B Version 2.00. Most of the transfers were done using queries to extract the data from the

PAPDB Version 1.00 and append it to tables in the PAP DB Version 2.00. SNL notes that t here is some order dependency on transferring the data and therefore, SNL created a VBA (Visual Basic for Applications) routine to execute the transfers in the proper order. This routine also deletes the

current contents of each P APDB Version 2.00 table before appending the data. The transfer of

the data from ‘ dbo ParameterRecor d ’ requires additional processing in VBA code to parse the

XML ‘ value ’ field and extract the numeric data from the string. A series of queries are used in6

the transfer of data from tables in PAPDB Version 1 to equivalent tables in PAPDB Version 2

. There ar e two tables that are not populated from the PAPDB Version 1.00: the

‘ DatabaseVersion ’ table, used to store the current version number of the database, and the Attributes table, which lists the names of the attributes that might be assigned to a parameter, such as the mean, median or constant value. SNL expects that the Attributes table will be used in the future to facilitate entering new parameter values or for display ing results.

3.5 ACCEPTANCE CRITERIA

SNL checked the transfers among the tables for comple teness using a series of queries and, for the parameter value data, two VBA routines. SNL designed the queries to compare the PAPDB Version 2.00 values to the corresponding values in the PAPDB Version 1.00 and to show only

records in the PAPDB Version 1.00 that are not matched in the PAPDB Version 2.00. Thus, if the tables transferred correctly the result would be an empty set. SNL notes that t he comparison of numeric values back to the values stored in the XML strings of ‘ dbo_parameterRecord ’ is problemati c because the numeric values were stored using several different formats, e.g. 1, 1.0, 1.00, 1.0e+00, etc. There were three formats that were used most frequently and VBA code was used to generate three XML strings from the PAPDB Version 2.00 data for each record of the dbo_parameterRecord using these formats. The refore, SNL concludes that the comparison in this case succeeds if any of the three generated XML string s exactly matches the original XML string. The Agency agrees with this assessment. SNL genera ted a report to display the strings that could not be matched in this way. A visual examination of t his report by the Agency found that the XML strings are equivalent.

3.6 TESTING M ETHODOLOGY

3.6.1 Migration from SQL to MySQL Server

SNL completed two major task s in the transfer process, 1) verification that the transfer procedures transfer the data, and 2) performing the transfer. The verification process actually involved a transfer of the data, followed by the deletion of some records from the various tables a nd execution of the various queries to show that the missing records are detected. Following the verification of the procedures the data are transferred once again and the same queries run to verify that no records were dropped during the transfer. It is a lso necessary to generate a report that shows the records in dbo_parameterRecords whose XML value string could not be matched are due to differences in immaterial factors (e.g., formatting , run dates).

3.6.2 Code Functionality

The Verification and Validation Plan /Validation Report (VVP) for the PAPDB (Kirchner 2011a)

describes how the functional requirements (Table 3 .1) described in the Requirements Document (Kirchner 2011b) will be tested, and the Agency found that it is in confo rmance with WIPP QA

NP 19

- 1 Re vision 12, Software Requirements

. The Validation Document (VD) d escribes the

success of the verification and validation tests. SNL use d th e utility program Test_ParamDB (Appendix 1 of Kirchner 2011a) to ve rify that the parameter data can be extracted from the PAPDB Version 2.00 using a FORTRAN application linked to a set of C routines. Similar7

routines were implemented in SDBREAD

_ LIB

. The PAPDB database was tested by SNL using the following environment:

Database Server OS: Solaris 11

Database Software : M ySQL 5.1.37

Database Name: ParamDB

The Solaris operating system was upgraded to V ersion 11 following the publication of the VVP. The version of MySQL was changed because Version 5.1.37 was provided with the installation of the operation system from Orac le. The FORTRAN code used by SNL to test the database was run on the Solaris platform. The application of service packs (whether to the OS or to the database software) or patches have no impact on the test cases.

While individual tests can be run and visu al inspection can be applied to verifying the results, the test framework provided by SNL has been developed to allow for inspection of the results of the tests by viewing an outpu t file produced by running all of the tests. There are three general types

o f testing that have be en performed: getting the database version, retrieving parameter data, and logging production retrievals.

Each major ‘functional area’ being verified is done through a conceptual grouping of test cases
into a ‘test’. Each ‘test’ is a grouping of one or more test cases that all support the verification of

a function supplied by the software. Each of the test cases for a particular test help support the testing of a major software function.

All tests have the same acceptance criterion : the expected output and the returned output must be identical in all relevant details. Differences in dates of log records, for example, are expected to be different. Neither spaces nor word case in output values are considered to be significant (relevan t). Any discrepancy that cannot be explained results in a test failure.

Table 3.6

. Software Requirements for the PAPDB Version 2 and Implementation Methods

Functional Requirements Implementation Method Verification

F1 The database mus t allow the user to retrieve the version number of the database. [PAPDB, Version 2.00 software must provide the version number of the database.] The table DatabaseVersion contains the current

version number of the database. Version is

retrieved and display ed using
Test_ParamDB utility.

F2 The database must be able to receive all relevant data from the PAPDB 1.00 [Documentation of the

successful transfer of all relevant data is described in the report documenting the transfer under AP

158 , (Kirchner 2011) ] The tables described in Table 1 of Kirchner (2012a) contain fields to store all relevant data. This requirement is verified by the procedures

described in AP

158 (Kirchner 2011)

F3 The database must provide the transfer of parameter

data from the PAPDB t o the codes or user interfaces that request them. [Numeric values for the parameters are stored in double precision fields. These data must be transferred, within the limits of floating point data, from the database to the applications requesting them.] My SQL provides the mysqlclient library for accessing data

. The data are passed as strings and must be converted to the appropriate type, double in the case of numerical values, using

the standard conversion function ‘ atof() ’

. SDBREAD LIB contains the code fo r accessing the PAPDB data for use by the PA codes. Verification is

done with the
Test_ParamDB utility.8

Table 3.6

. Software Requirements for the PAPDB Version 2 and Implementation Methods

Functional Requirements Implementation Method Verification

F4 The database must allow storage and retrieval of data by material, property, analysis and analysis revision, and code. [The goal is to be able to re trieve parameter values used by a code in an analysis. Queries which do not properly specify the retrieval of a parameter should fail to return a record set.] The specified keys are used, as appropriate, in

the tables listed in Table 1 of Kirchner (2012a)

. Verification of

retrieval is done

with the Test ParamDB utility.

F5 The database must allow for logging of retrievals by analysis, analysis revision, and code. [Logging should occur only for the execution of ‘ official ’ PAs, i.e. those conducted under th e WIPP QA program and defined by an analysis plan. Access should be granted for unofficial use but no logging performed. Control of logging will be managed by the interface routines like those in SDBREAD_L1B (Gilkey 20 12 ) The table Analysis Retrieval Histo ry is designed to record the export of data from the PAPDB. The control of the logging resides with the code interface, typically SDBREAD LIB. It is equivalent to the table dbo_Analysis Retrieval History in the PAPDB Version 1.00 except that it also record s the revision number of the analysis. Verification of logging is done

with the Test ParamDB utility.

Attributes

– Security Design/Implementation Method

A1 The database software must control write access to the data tables using passwords. The database is defined as a MySQL schema. [User accounts in the

database and the account used to access the database through code must be assigned passwords.

Read

- only access should be provided by default and write rights should be granted only to those users

authori zed to make changes or additions to the database. The database server computer is protected using passwords as required by Sandia National

Laboratories. MySQL controls access to data using user passwords and the delegatio n of rights to the

users. The P APDB database administrator has exclusive permission to create users and to assign privileges to the users. Verification of

control via MySQL user privileges is done

with Test

ParamDB.

3.6.3 Test Cases

SNL designed n ine (9) test cases in which all or some of the following are specified : material,
property, analysis, analysis revision number, code, and code version. The version number of the

database is output for every test. The test display s the version number of the database, the attributes assigned to the param eter for a specific analysis and analysis revision, discrete values assigned to that version's parameter's distribution, the documents associated with the parameter, and the analyses where the various versions of the parameter are first defined. These data correspond to the information displayed in the parameter viewer application used with Version

1.00 of the database. The test cases are:

Test Case 1

- Display the informat ion for a parameter having no defined di stribution

Test Case 2

- Display the informati on for a model parameter having a distribution with

standard statistical parameters ( e.g.
, a distribution other than a Student, Delta or

Cumulative )

.

Test Case 3

- D isplay the information for a parameter having two sets of values within one analysis

Test Ca se 4

- Display the information for a parameter having a Student distribution9

Test Case 5

- Display the information for a parameter having a Delta distribution

Test Case 6

- Display the information for a parameter having a Cumulative Distribution

Test Case 7

- Display that nothing is returned for a parameter having a non

- existent

material :property

Test Case 8

- Show that logging a record identical to one existing in Analysis Retrieval History updates the timestamp

.

Test Case 9

- Show that a user not having inserti on and deletion privileges cannot delete

or insert a record into the Analysis Retrieval History table

3.6.4 Execution of Tests

SNL conducted the t ests using the FORTRAN code Test_ParamDB and the scripts RunTests.csh and RunTest9.csh. The source code, executabl e file, makefile (build script), a copy of the database as a sqldump file, the Test_ParamDB.out file and the test scripts were stored in the CVS repository $CVSLIB/Databases/ParamDB

. Several of these output files were reviewed by the Agency during the onsi te review in October 2012.

The output of each test was stored in the file Test_ParamDB.out. This file becomes the expected output for future regression tests. The test outputs were verified by SNL by inspecting database tables using the WIPP Parameter Vie wer (a web application) to determine if the correct results

were returned. The Agency verified a subset of the se test results with the WIPP Parameter

Viewer and all of the hardcopy output

. Although the VVP stated that the Version 1.00 PAPDB

would be used f

or verification, SNL used Version 2.00 because: 1) the design of the Parameter

Viewer for the Version 1.00 database was found to omit some documents associated with the parameters, and 2) the complete transfer of data from the Version 1.00 to the Version 2 .00

database was verified and documented in Kirchner (2012).

3.7 T EST R ESULTS

While individual tests can be run and visual inspection can be applied to verifying the results, the test framework provided has been developed to allow for inspection of the resul ts of the tests by viewing an output file produced by running all of the tests. The results of the tests were compared to the data stored in the database and, for sample values for some distributions, viewing the data

from the Distribution Data table using the Parameter Transfer database described in Kirchner (2012).

Failure in the transfer of data from the PAPDB Version 1.00 would result in missing records in

one or more of the PAPDB Version 2.00 tables or having data in corresponding records that do

not match. Potentially missing or erroneous records are identified using a series of ‘ Check ’ queries. SNL verified that these Check queries function ed correctly by 1) transferring the data

, and 2 ) deleting one or more records and modifying values in the remain ing records from each of

the tables listed in Table 6 and 3 in Kirchner (2012a). Since the record set of the Check quer ies

were all empty the transfer was determined by the Agency to be successful and was is indicated10

in the Check Transfer form documenting that PAPDB Version 2.00 meets the requirements listed

in the VVP.11 

### 4.0 SUMMARY AND C ONCLUSIONS

The primary objectives of the Agency’s review of the PAPDB database migration to the new platform are to assess the transcription accuracy, documentation, and trac eability of changes to the parameter values, distributions and metadata

. The PAPDB parameter database and associated metadata are maintained in Carlsbad, New Mexico, by SNL. Hard copies of documentation supporting the parameters and their values are mainta ined in SNL’s WIPP Records Center in Carlsbad. The supporting documents are tracked by SNL’s Electronic Record Management System (ERMS), through which a unique identifying number is assigned to each document.

The Agency’s review of the adequacy of SNL’s implementation and testing of the database migration was initiated in October 2012 with an on

- site review of the PAPDB and its PA code

interfaces. The Agency has reviewed the PAPDB life

- cycle documentation (e.g., Requirements Document, Design Document, Ve rification and Validation Plan) to determine whether the

PAPDB migration to a new platform meet the requirements specified in § 194.22, § 194.23 and ASME NQA

- 2a

- 1990 addenda, part 2.7

. In addition to the life

- cycle documentation , the Agency

reviewed several migration reports to further evaluate the migration testing procedures and

results (Kirchner and Long 2012 ; Kirchner 2011).

Based upon these reviews, the Agency found that the PADB verification tests adequately test all the functional requirements necessa ry for PAPDB Version 2.00 to effectively support PA, and that PAPDB Version 2.00 has successfully passed the tests

. Furthermore, the Agency determined that SNL has developed documentation that is consistent with life

- cycle and testing requirements

outlined in § 194.22, § 194.23 and ASME NQA

- 2a

- 1990 addenda, part 2.7. The Agency therefore concludes that PAPDB Version 2.00 is in compliance with §194.22, §194.23 and ASME NQA

2a

- 1990 addenda, part 2.7 and that PAPDB Version 2.00 is approved for WIPP PA use in PA

calculations.12 

### 5.0 R EFERENCES

Chavez, M. J. 2006. NP 19

- 1 Revision 12, Software Requirements. Sandia National Laboratories, Carlsbad, NM. ERMS #543742.

Gilkey, A. P. 2012. WIPP PA User's Manual for SDBREAD_LIB Version 4.00, Sandia National Laboratories, Carl sbad, NM. ERMS # 556534.

Kirchner, T. B., and J. Long. 2012. Report on the Migration of the WIPP PA Parameter Database from SQL Server to MySQL. Sandia National Laboratories, Carlsbad, NM. ERMS #557128.

Kirchner, T. B. 2011. Analysis Plan for Migration of the WIPP PA Parameter Database from SQL Server to MySQL. Carlsbad, NM: Sandia National Laboratories. ERMS # 556953.

Kirchner, T. B. 2011a. Software QA Plan Criteria PAPDB Version 2.00 Sandia National Laboratories, Carlsbad, NM. ERMS #555997.

Kirchner, T . B. 2011b. The PAPDB Version 2.00 Requirements Document. Sandia National Laboratories, Carlsbad, NM. ERMS #555998.

Kirchner, T. B. 2011c. The PAPDB Version 2.00 Verification and Validation Plan. Sandia National Laboratories, Carlsbad, NM. ERMS #556729.

Kirchner, T. B. 2012. Design Document PAPDB Version 2.00 Sandia National Laboratories, Carlsbad, NM. ERMS #556730.

Kirchner, T. B. 2012a. Val i dation Document PAPDB Version 2.00 Sandia National Laboratories, Carlsbad, NM. ERMS #556001.

Kirchner, T. B. 201 2b. Installation and Checkout PAPDB Version 2.00 Sandia National Laboratories, Carlsbad, NM. ERMS #556002.

Kirkes, G.R. and C.G. Herrick. 2006. Analysis Plan for the Modification of the Waste Shear Strength Parameter and Direct Brine Release Parameters, A P

- 131

. Sandia National Laboratories, Carlsbad, NM. ERMS 541530.