APPENDIX B

REVIEW OF WIPP PERFORMANCE ASSESSMENT 

COMPUTERIZED DATABASE (PAPDB) MIGRATION
i

TABLE OF CONTENTS

Executive Summary........................................................................................................................ ii

1.0 Introduction..........................................................................................................................1

2.0 General Approach ................................................................................................................3

3.0 Review of the PAPDB Migration ........................................................................................4

3.1 Rationale for Database Migration and Update.........................................................4

3.2 Migration Objectives ...............................................................................................5

3.3 Approach..................................................................................................................5

3.4 Software ...................................................................................................................5

3.5 Acceptance Criteria..................................................................................................6

3.6 Testing Methodology ...............................................................................................6

3.6.1 Migration from SQL to MySQL Server.......................................................6

3.6.2 Code Functionality.......................................................................................6

3.6.3 Test Cases ....................................................................................................8

3.6.4 Execution of Tests........................................................................................9

3.7 Test Results..............................................................................................................9

4.0 Summary and Conclusions.................................................................................................11

5.0 References..........................................................................................................................12
ii

EXECUTIVE SUMMARY

The ability of the DOE WIPP facility to continue to meet the certification requirements of the 

EPA is demonstrated in part through the use of a series of performance assessment (PA) 

computer codes. DOE must demonstrate on an ongoing basis that PA computer software is in 

compliance with regulations outlined in Section §194.23 – Models and Computer Codes and in 

ASME NQA-2a-1990 addenda, part 2.7. Since the Agency’s certification of the DOE WIPP 

Compliance Certification Application (CCA), DOE has added computer hardware and upgraded 

the computer software. In order to maintain compliance with Sections §194.22, §194.23 and 

ASME NQA-2a-1990 addenda, part 2.7, DOE is required to conduct testing on the computer 

codes to ensure that they still function properly on new hardware and software.

This appendix describes the results of the U.S. Environmental Protection Agency’s (EPA or the 

Agency) review of Performance Assessment Parameter Database (PAPDB) migration, design

modifications and testing activities performed by Sandia National Laboratories (SNL) in support 

of their ongoing Performance Assessment (PA) activities at the Waste Isolation Pilot Plant 

(WIPP).

In 2011, SNL migrated the PAPDB (Version 1.0) used in the WIPP compliance calculations from

the Microsoft SQL Server on a Windows Server to the PAPDB (Version 2.00) on the MySQL 

Server on Solaris. The purpose of the migration is to facilitate access to the parameter data from 

the PA codes on Solaris, Linux and VMS. In addition, the design of the database was also

modified to improve its ease of use.

SNL defines a parameter as any value or distribution of values used directly or indirectly as input 

to a PA. The PAPDB contains the parameters used in WIPP PAs, the names of documents related 

to those parameters, the names of the codes that use the parameters, and a log of the parameters 

that were exported to the various codes during analyses. In the PAPDB a parameter may be 

represented by a single value or assigned a distribution which represents epistemic uncertainty. 

The values of parameters can change through time as new information becomes available. The 

database contains the different versions of the parameter(s) which are identified by a version 

number, by the analysis where the parameter was first defined, and by the effective date for its

first use. Specific versions of a parameter can be associated with one or more analyses. The 

purpose of the database is to securely store the parameter data used in the PA analyses, to allow 

access to that data directly from the applications that need those data, and to log the usage of the 

data. In addition, the PAPDB is designed to access the data from a user interface to facilitate 

querying the database to document the parameter values used in the various analyses

The primary objectives of the Agency’s review of the PAPDB database migration to the new 

platform are to assess the transcription accuracy, documentation, and traceability of changes to 

the parameter values, distributions and metadata. The PAPDB parameter database and associated 

metadata are maintained in Carlsbad, New Mexico, by SNL. Hard copies of documentation 

supporting the parameters and their values are maintained in SNL’s WIPP Records Center in 

Carlsbad. The supporting documents are tracked by SNL’s Electronic Record Management 

System (ERMS), through which a unique identifying number is assigned to each document. 
iii

The Agency has reviewed the PAPDB life-cycle documentation (e.g., Requirements Document, 

Design Document, Verification and Validation Plan) to determine whether the PAPDB migration 

to a new platform meets the requirements specified in §194.22, §194.23 and ASME NQA-2a1990 addenda, part 2.7. In addition to the life-cycle documentation, the Agency reviewed several 

migration reports to further evaluate the migration testing procedures and results (Kirchner and 

Long 2012; Kirchner 2011).

Based upon these reviews, the Agency found that the PAPDB verification tests adequately test all 

the functional requirements necessary for PAPDB Version 2.00 to effectively support PA, and 

that PAPDB Version 2.00 has successfully passed the tests. Furthermore, the Agency determined 

that SNL has developed documentation that is consistent with life-cycle and testing requirements 

outlined in §194.22, §194.23 and ASME NQA-2a-1990 addenda, part 2.7. The Agency therefore 

concludes that PAPDB Version 2.00 is in compliance with §194.22, §194.23 and ASME NQA2a-1990 addenda, part 2.7 and that PAPDB Version 2.00 is approved for WIPP PA use in PA 

calculations.
1

1.0 INTRODUCTION

This appendix describes the results of the U.S. Environmental Protection Agency’s (EPA or the 

Agency) review of Performance Assessment Parameter Database (PAPDB) migration, design 

modifications and testing activities performed by Sandia National Laboratories (SNL) in support 

of their ongoing Performance Assessment (PA) activities at the Waste Isolation Pilot Plant 

(WIPP).

In 2011, SNL developed new database software, called PAPDB Version 2.00, to replace the 

previous PAPDB Version 1.00, and migrated the parameter values and associated metadata from 

Version 1.00 to Version 2.00. The database structure was modified to make it easier to use and to 

facilitate parameter data access from the PA codes on Solaris, Linux, and VMS platforms. The 

new database resides on a MySQL Server and is physically located on an Intel-based server 

running Solaris 11 (Kirchner and Long 2012). The database migration process is more fully 

described by Kirchner (2011).

A primary improvement in the new database is the ability to accurately retrieve parameter values 

used in earlier PAs. PAPDB Version 2.00 is designed to default to the parameter value most 

recently entered into the database, and to automatically log in the analysis that used the parameter 

value. However, a previous value can also be returned by specifying an earlier analysis. A similar 

functionality was provided in PAPDB Version 1.00, but did not work properly (Kirchner 2011).

In addition, distributed parameters are now assigned a default value when the distribution is not 

to be sampled. For example, when assigning three-dimensional, isotropic permeability values to a 

material, only the value of PRMX_LOG is randomly sampled using the PRELHS code. The 

constant default values for PRMY_LOG and PRMZ_LOG are taken from the database as 

placeholders and are subsequently reassigned the same value as PRMX_LOG in the BRAGFLO 

ALGEBRA file. 

The methodology for retrieving parameter values from the database remains essentially 

unchanged. The MATSET code retrieves the default value for every parameter in the analysis. 

Parameter values that are to be sampled are identified in the input files to the PRELHS code, 

which supplies that information to the LHS code. The LHS code then retrieves 100 sample 

values per replicate from the database and supplies them to the post LHS code. If a parameter has 

been retrieved by LHS, the POSTLHS code replaces the default parameter value retrieved by 

MATSET with the parameter value retrieved by LHS. In a subsequent step, the remaining default 

values retrieved by MATSET for distributed parameters are replaced by the desired value in the 

ALGEBRA1 file, as described in the example above. For constant parameters, the default value 

is the constant value and remains unchanged. Although the default values for distributed 

parameters are typically the means or medians of the distributions, any value can be used as the 

default value. 

The PAPDB Version 2.00 database tables are constructed to store the data residing in PAPDB 

Version 1.00. An Access database, ParamTransfer.mdb, was used to transfer the data from 

PAPDB Version 1.00 to PAPDB Version 2.00. Most of the transfers were done using queries to 

extract the data from the PAPDB Version 1.00 and append it to tables in the PAPDB Version 
2

2.00. The transfers were checked for completeness using a series of queries and, for the 

parameter value data, two VBA (Visual Basic for Applications) routines. The queries were 

designed to compare the PAPDB Version 2.00 values with the corresponding values in PAPDB 

Version 1.00 and to show only records in the PAPDB Version 1.00 that were not matched in the 

PAPDB Version 2.00. Thus, the desired outcome was an empty data set. 

Verification of the transfer process involved a transfer of the data followed by the deletion of 

selected records from the various tables and execution of the various queries to show that the 

missing records were detected. Following the verification of the procedures, the data were 

transferred once again and the same queries run to verify that no records were dropped during the 

transfer. It was also necessary to generate a report that showed the records whose value string

could not be matched due to differences in formatting. The records shown in that report were 

compared and verified to be equivalent.

In addition to the records transferred from PAPDB Version 1.00, two sets of records were 

developed that were missing from the original database. The design of the PAPDB Version 1.00 

did not allow for multiple analyses being run under a single AP, as was done for AP-132. It 

appears to have kept only the records from the most recent retrieval of parameters. Furthermore, 

the retrieval of parameters by MATSET for use in NUTS seems to never have been recorded. To 

capture these usages of parameters, a series of queries were run to insert the missing records.

The data were transferred following verification of the transfer procedures. SNL and the Agency 

verified an accurate data transfer by reviewing the comparison reports, as documented by 

Kirchner and Long (2012). Problems found by SNL in initial data transfers were corrected and in 

the final transfer, all of the comparison tests were successful except one, where the data did not 

exactly match due to formatting differences. Those test results were reviewed manually and SNL 

concluded that the values of the two data sets were found to be equivalent. The Agency agrees 

with this assessment.

The remaining sections present the approach that the Agency followed to determine whether 

PAPDB Version 2.00 is in compliance with §194.22, §194.23 and ASME NQA-2a-1990 

addenda, part 2.7. 
3

2.0 GENERAL APPROACH

To demonstrate that computer software is in compliance with disposal regulations outlined in 

§194.22, SNL established a life-cycle management process for software used to support their PA

activities. Their qualification approach for the software follows the life-cycle phases outlined in 

ASME NQA-2a-1990 addenda, part 2.7, as follows:

Planning

Requirements

Design

Implementation

Validation

Installation and Checkout

Maintenance

Retirement

SNL implements the life-cycle phases using an iterative or sequential approach and each phase is 

supported with relevant documentation (e.g., Design Document, Verification Plan, and 

Implementation Document).

The Agency’s review of the adequacy of SNL’s implementation and testing of the database 

migration was initiated in October 2012 with an on-site review of the PAPDB and its PA code 

interfaces. SNL maintains the parameter database and associated metadata in Carlsbad, New 

Mexico. Hard copies of documentation supporting the parameters and their values are retained in 

SNL’s WIPP Records Center in Carlsbad. SNL tracks the supporting documents by an Electronic 

Record Management System (ERMS) in which a unique identifying number is assigned to each 

document.

The primary objectives of the Agency’s review of the database migration to the new platform 

were to assess the adequacy of the documentation, traceability of the modifications and reliability 

of the testing. The scope of the review was primarily focused upon reviewing the following

documentation for PAPDB (Version 2):

Analysis Plan (Kirchner 2011)

Software Quality Assurance Plan (Kirchner 2011a)

Requirements Document (Kirchner 2011b)

Verification Validation Plan (Kirchner 2011c)

Design Document (Kirchner 2012)

Validation Document (Kirchner 2012a)

Installation and Checkout (Kirchner 2012b)

Migration Report (Kirchner and Long 2012)

The transcription accuracy of the migration and the traceability of changes to the parameter 

values and metadata that were made for the PCS-2012 PA were also evaluated (Appendix A). 
4

3.0 REVIEW OF THE PAPDB MIGRATION

3.1 RATIONALE FOR DATABASE MIGRATION AND UPDATE

In 2011, the Parameter Database (PAPDB Version 1.0) was revised to Version 2.0 and migrated 

from the Microsoft SQL on a Windows Server to the MySQL software running Solaris 11. Prior 

to being migrated to MySQL, the parameter database (PAPDB) was physically located on an 

Intel-based server also running Solaris 11, tgw.sandia.gov. SNL migrated the database to 

facilitate access to the parameter data from the PA codes located on Solaris, Linux and VMS. 

The primary function of PAPDB is to supply parameter values for use in PA and PAPDB

contains data values, associated codes, references for relevant documents, usage and additional 

information documenting the parameters used in PA. These data are exported to the other PA 

codes via routines in the SDBREAD_LIB. In PAPDB Version 1.00 the numeric data was 

encoded for a parameter, such as the mean, median and standard deviation, into an XMLformatted string. This was done to facilitate the display of the data via a web browser, which was 

the original goal for the database. SNL, however, notes that this design complicates the interface 

to the PA codes since the desired values must be unpacked from the XML string and converted 

from text to numeric data. 

To alleviate the complicated design, SNL redesigned PAPDB Version 1.00 into Version 2.00 

which has no stored procedures or queries. In Version 2.00, the business rules that had been 

implemented as stored procedures in Version 1.00 have either been eliminated through the 

redesign of the database or have been implemented in the revision of SDBREAD_LIB. For 

example, the logging of parameter export by analysis and code is now the responsibility of 

SDBREAD_LIB rather than implemented as a stored procedure. Thus, the PAPDB Version 2.00 

provides storage for the data but has no inherent functionality. 

The specifications for the PAPDB Version 1.00 state that the parameter values obtained via 

SDBREAD_LIB should default to the most recent version unless an earlier version is specified 

(PAPDB Version 1.00 Requirements Document). SDBREAD_LIB contains the routines that are 

the interface between the performance assessment (PA) codes and the database. However, SNL 

notes that this functionality appears to be flawed since the ability to specify earlier versions does 

not seem to work properly (Kirchner 2011). Special analyses run by SNL to support PA over the 

last few years often require new parameter values, with subsequent return to the parameters of 

the last PA baseline calculation. Thus, the typical approach that SNL has followed has been to 1) 

enter the new values of some parameters for the PA, 2) run the PA, and 3) restore the values from 

the last baseline calculation by re-entering them. This process is unproductive because it requires 

a number of changes to the database, adds redundant data to the database, and increases the 

chance of making an error when restoring baseline values. Furthermore, SNL describes a 

problem that arises in the ‘AnalysisID’ field of the PAPDB (Version 1)(Kirchner 2011). This 

field reflects the analysis when parameters are first defined, but in some cases those parameters 

were not used in that analysis, e.g. the lower limit of the distribution assigned to 

BOREHOLE:TAUFAIL was redefined in AP-131 (Kirkes and Herrick 2006) and entered into the 

database but the PA for AP-131 did not use those new values. In this case, the code that used 
5

BOREHOLE:TAUFAIL, PRELHS, was not run and instead the LHS sampled values from a 

previous analysis were used in the AP-131 PA. As noted by SNL, no record was entered into the 

‘AnalysisRetrievalHistory’ and thus the PAPDB Version 1 database could not be used to 

determine which version of the parameter was really used in the PA. 

3.2 MIGRATION OBJECTIVES

SNL designed the MySQL version of the PAPDB (PAPBD Version 2.00) to store numeric values 

associated with parameters in a numeric field that allow the codes to specify a specific version of 

a parameter value to be returned by specifying an analysis. 

To allow a determination of which version of the parameter was used in the PA SNL added a

new table to map from an analysis name to a version for each parameter used in an analysis. In 

addition, parameters assigned distributions are assigned a ‘value’ property in the PAPDB Version 

2.00 that are used as the default value when the distribution is not to be sampled. In PAPDB 

Version 1.00 the median is used as the default value for most distributions and the mode for 

triangular distributions. SNL notes that this has the consequence that the value assigned to the 

median has not always been the true median of the distribution. SNL also indicates that this 

situation has no impact on the distribution sampled by LHS because for the majority of the 

distributions, excluding the triangular, cumulative and Student’s distributions, it is only the 

maximum and minimum values that are used to specify the distributions. SNL added the default 

values in the PAPDB Version 2.00 to remedy this issue. 

3.3 APPROACH

The PAPDB Version 2.00 database tables are constructed to store the data residing in the 

PAPDB Version 1.00. SNL, however, simplified the structure of the database by eliminating the 

many ‘index’ tables, such as the table dbo Material which lists the material name along with a 

numeric index associated with the material name. The number of tables was reduced from 41 to 

14 for inclusion in PAPDB Version 2. A number of the tables in PAPDB Version 1 were meant 

for functions that were never implemented. Other tables stored values that were always the same, 

or which were never used in the parameters viewer or to retrieve values for the codes. SNL did 

not transfer these tables to PAPDB Version 2. In some cases, SNL combined two or more tables

into one.

3.4 SOFTWARE

SNL used an Access database, ParamTransfer.mdb, to transfer the data from PAPDB Version 1 to

PAPDB Version 2.00. Most of the transfers were done using queries to extract the data from the 

PAPDB Version 1.00 and append it to tables in the PAPDB Version 2.00. SNL notes that there is 

some order dependency on transferring the data and therefore, SNL created a VBA (Visual Basic 

for Applications) routine to execute the transfers in the proper order. This routine also deletes the 

current contents of each PAPDB Version 2.00 table before appending the data. The transfer of 

the data from ‘dbo ParameterRecord’ requires additional processing in VBA code to parse the 

XML ‘value’ field and extract the numeric data from the string. A series of queries are used in 
6

the transfer of data from tables in PAPDB Version 1 to equivalent tables in PAPDB Version 2. 

There are two tables that are not populated from the PAPDB Version 1.00: the 

‘DatabaseVersion’ table, used to store the current version number of the database, and the 

Attributes table, which lists the names of the attributes that might be assigned to a parameter,

such as the mean, median or constant value. SNL expects that the Attributes table will be used in 

the future to facilitate entering new parameter values or for displaying results.

3.5 ACCEPTANCE CRITERIA

SNL checked the transfers among the tables for completeness using a series of queries and, for 

the parameter value data, two VBA routines. SNL designed the queries to compare the PAPDB 

Version 2.00 values to the corresponding values in the PAPDB Version 1.00 and to show only 

records in the PAPDB Version 1.00 that are not matched in the PAPDB Version 2.00. Thus, if 

the tables transferred correctly the result would be an empty set. SNL notes that the comparison 

of numeric values back to the values stored in the XML strings of ‘dbo_parameterRecord’ is 

problematic because the numeric values were stored using several different formats, e.g. 1, 1.0, 

1.00, 1.0e+00, etc. There were three formats that were used most frequently and VBA code was 

used to generate three XML strings from the PAPDB Version 2.00 data for each record of the 

dbo_parameterRecord using these formats. Therefore, SNL concludes that the comparison in this 

case succeeds if any of the three generated XML strings exactly matches the original XML string. 

The Agency agrees with this assessment. SNL generated a report to display the strings that could 

not be matched in this way. A visual examination of this report by the Agency found that the 

XML strings are equivalent. 

3.6 TESTING METHODOLOGY

3.6.1 Migration from SQL to MySQL Server

SNL completed two major tasks in the transfer process, 1) verification that the transfer procedures 

transfer the data, and 2) performing the transfer. The verification process actually involved a transfer 

of the data, followed by the deletion of some records from the various tables and execution of the 

various queries to show that the missing records are detected. Following the verification of the 

procedures the data are transferred once again and the same queries run to verify that no records were 

dropped during the transfer. It is also necessary to generate a report that shows the records in 

dbo_parameterRecords whose XML value string could not be matched are due to differences in 

immaterial factors (e.g., formatting, run dates).

3.6.2 Code Functionality

The Verification and Validation Plan/Validation Report (VVP) for the PAPDB (Kirchner 2011a)

describes how the functional requirements (Table 3.1) described in the Requirements Document 

(Kirchner 2011b) will be tested, and the Agency found that it is in conformance with WIPP QA 

NP 19-1 Revision 12, Software Requirements. The Validation Document (VD) describes the 

success of the verification and validation tests. SNL used the utility program Test_ParamDB 

(Appendix 1 of Kirchner 2011a) to verify that the parameter data can be extracted from the

PAPDB Version 2.00 using a FORTRAN application linked to a set of C routines. Similar 
7

routines were implemented in SDBREAD_LIB. The PAPDB database was tested by SNL using 

the following environment:

Database Server OS: Solaris 11

Database Software: MySQL 5.1.37

Database Name: ParamDB

The Solaris operating system was upgraded to Version 11 following the publication of the VVP. 

The version of MySQL was changed because Version 5.1.37 was provided with the installation 

of the operation system from Oracle. The FORTRAN code used by SNL to test the database was 

run on the Solaris platform. The application of service packs (whether to the OS or to the 

database software) or patches have no impact on the test cases.

While individual tests can be run and visual inspection can be applied to verifying the results, the 

test framework provided by SNL has been developed to allow for inspection of the results of the 

tests by viewing an output file produced by running all of the tests. There are three general types 

of testing that have been performed: getting the database version, retrieving parameter data, and 

logging production retrievals. 

Each major ‘functional area’ being verified is done through a conceptual grouping of test cases 

into a ‘test’. Each ‘test’ is a grouping of one or more test cases that all support the verification of 

a function supplied by the software. Each of the test cases for a particular test help support the 

testing of a major software function.

All tests have the same acceptance criterion: the expected output and the returned output must be 

identical in all relevant details. Differences in dates of log records, for example, are expected to 

be different. Neither spaces nor word case in output values are considered to be significant 

(relevant). Any discrepancy that cannot be explained results in a test failure.

Table 3.6. Software Requirements for the PAPDB Version 2 and Implementation Methods

Functional Requirements Implementation Method Verification

F1

The database must allow the user to retrieve the 

version number of the database. [PAPDB, Version 

2.00 software must provide the version number of 

the database.]

The table DatabaseVersion contains the current 

version number of the database.

Version is

retrieved and 

displayed using 

Test_ParamDB 

utility.

F2

The database must be able to receive all relevant 

data from the PAPDB 1.00 [Documentation of the 

successful transfer of all relevant data is described 

in the report documenting the transfer under AP158, (Kirchner 2011)]

The tables described in Table 1 of Kirchner 

(2012a) contain fields to store all relevant data.

This requirement 

is verified by the 

procedures 

described in AP158

(Kirchner 2011)

F3

The database must provide the transfer of parameter 

data from the PAPDB to the codes or user 

interfaces that request them. [Numeric values for 

the parameters are stored in double precision fields. 

These data must be transferred, within the limits of 

floating point data, from the database to the 

applications requesting them.]

MySQL provides the mysqlclient library for 

accessing data. The data are passed as strings 

and must be converted to the appropriate type, 

double in the case of numerical values, using 

the standard conversion function ‘atof()’. 

SDBREAD LIB contains the code for accessing 

the PAPDB data for use by the PA codes.

Verification is 

done with the 

Test_ParamDB 

utility.
8

Table 3.6. Software Requirements for the PAPDB Version 2 and Implementation Methods

Functional Requirements Implementation Method Verification

F4

The database must allow storage and retrieval of 

data by material, property, analysis and analysis 

revision, and code.

[The goal is to be able to retrieve parameter values 

used by a code in an analysis. Queries which do not 

properly specify the retrieval of a parameter should 

fail to return a record set.]

The specified keys are used, as appropriate, in 

the tables listed in Table 1 of Kirchner (2012a).

Verification of 

retrieval is done 

with the Test 

ParamDB utility.

F5

The database must allow for logging of retrievals by 

analysis, analysis revision, and code. [Logging 

should occur only for the execution of ‘official’

PAs, i.e. those conducted under the WIPP QA 

program and defined by an analysis plan. Access 

should be granted for unofficial use but no logging 

performed. Control of logging will be managed by 

the interface routines like those in SDBREAD_L1B 

(Gilkey 2012)

The table Analysis Retrieval History is 

designed to record the export of data from the 

PAPDB. The control of the logging resides 

with the code interface, typically SDBREAD 

LIB. It is equivalent to the table dbo_Analysis 

Retrieval History in the PAPDB Version 1.00 

except that it also records the revision number 

of the analysis. 

Verification of 

logging is done 

with the Test 

ParamDB utility.

Attributes – Security Design/Implementation Method

A1

The database software must control write access to 

the data tables using passwords. The database is 

defined as a MySQL schema. [User accounts in the 

database and the account used to access the 

database through code must be assigned passwords. 

Read-only access should be provided by default and 

write rights should be granted only to those users 

authorized to make changes or additions to the 

database. The database server computer is protected 

using passwords as required by Sandia National 

Laboratories.

MySQL controls access to data using user 

passwords and the delegation of rights to the 

users. The PAPDB database administrator has 

exclusive permission to create users and to 

assign privileges to the users.

Verification of 

control via 

MySQL user 

privileges is done 

with Test 

ParamDB.

3.6.3 Test Cases

SNL designed nine (9) test cases in which all or some of the following are specified: material, 

property, analysis, analysis revision number, code, and code version. The version number of the

database is output for every test. The test displays the version number of the database, the 

attributes assigned to the parameter for a specific analysis and analysis revision, discrete values 

assigned to that version's parameter's distribution, the documents associated with the parameter, 

and the analyses where the various versions of the parameter are first defined. These data

correspond to the information displayed in the parameter viewer application used with Version

1.00 of the database. The test cases are:

Test Case 1- Display the information for a parameter having no defined distribution

Test Case 2- Display the information for a model parameter having a distribution with 

standard statistical parameters (e.g., a distribution other than a Student, Delta or

Cumulative).

Test Case 3- Display the information for a parameter having two sets of values within one

analysis

Test Case 4- Display the information for a parameter having a Student distribution
9

Test Case 5- Display the information for a parameter having a Delta distribution

Test Case 6- Display the information for a parameter having a Cumulative Distribution

Test Case 7- Display that nothing is returned for a parameter having a non-existent

material:property

Test Case 8- Show that logging a record identical to one existing in Analysis Retrieval

History updates the timestamp.

Test Case 9- Show that a user not having insertion and deletion privileges cannot delete 

or insert a record into the Analysis Retrieval History table

3.6.4 Execution of Tests

SNL conducted the tests using the FORTRAN code Test_ParamDB and the scripts RunTests.csh 

and RunTest9.csh. The source code, executable file, makefile (build script), a copy of the 

database as a sqldump file, the Test_ParamDB.out file and the test scripts were stored in the CVS 

repository $CVSLIB/Databases/ParamDB. Several of these output files were reviewed by the 

Agency during the onsite review in October 2012.

The output of each test was stored in the file Test_ParamDB.out. This file becomes the expected 

output for future regression tests. The test outputs were verified by SNL by inspecting database 

tables using the WIPP Parameter Viewer (a web application) to determine if the correct results 

were returned. The Agency verified a subset of these test results with the WIPP Parameter 

Viewer and all of the hardcopy output. Although the VVP stated that the Version 1.00 PAPDB 

would be used for verification, SNL used Version 2.00 because: 1) the design of the Parameter 

Viewer for the Version 1.00 database was found to omit some documents associated with the 

parameters, and 2) the complete transfer of data from the Version 1.00 to the Version 2.00 

database was verified and documented in Kirchner (2012).

3.7 TEST RESULTS

While individual tests can be run and visual inspection can be applied to verifying the results, the 

test framework provided has been developed to allow for inspection of the results of the tests by 

viewing an output file produced by running all of the tests. The results of the tests were compared

to the data stored in the database and, for sample values for some distributions, viewing the data 

from the Distribution Data table using the Parameter Transfer database described in Kirchner 

(2012).

Failure in the transfer of data from the PAPDB Version 1.00 would result in missing records in 

one or more of the PAPDB Version 2.00 tables or having data in corresponding records that do 

not match. Potentially missing or erroneous records are identified using a series of ‘Check’

queries. SNL verified that these Check queries functioned correctly by 1) transferring the data, 

and 2) deleting one or more records and modifying values in the remaining records from each of 

the tables listed in Table 6 and 3 in Kirchner (2012a). Since the record set of the Check queries 

were all empty the transfer was determined by the Agency to be successful and was is indicated 
10

in the Check Transfer form documenting that PAPDB Version 2.00 meets the requirements listed 

in the VVP.
11

4.0 SUMMARY AND CONCLUSIONS

The primary objectives of the Agency’s review of the PAPDB database migration to the new 

platform are to assess the transcription accuracy, documentation, and traceability of changes to 

the parameter values, distributions and metadata. The PAPDB parameter database and associated 

metadata are maintained in Carlsbad, New Mexico, by SNL. Hard copies of documentation 

supporting the parameters and their values are maintained in SNL’s WIPP Records Center in 

Carlsbad. The supporting documents are tracked by SNL’s Electronic Record Management 

System (ERMS), through which a unique identifying number is assigned to each document. 

The Agency’s review of the adequacy of SNL’s implementation and testing of the database 

migration was initiated in October 2012 with an on-site review of the PAPDB and its PA code 

interfaces. The Agency has reviewed the PAPDB life-cycle documentation (e.g., Requirements 

Document, Design Document, Verification and Validation Plan) to determine whether the 

PAPDB migration to a new platform meet the requirements specified in §194.22, §194.23 and 

ASME NQA-2a-1990 addenda, part 2.7. In addition to the life-cycle documentation, the Agency 

reviewed several migration reports to further evaluate the migration testing procedures and 

results (Kirchner and Long 2012; Kirchner 2011).

Based upon these reviews, the Agency found that the PADB verification tests adequately test all 

the functional requirements necessary for PAPDB Version 2.00 to effectively support PA, and 

that PAPDB Version 2.00 has successfully passed the tests. Furthermore, the Agency determined 

that SNL has developed documentation that is consistent with life-cycle and testing requirements 

outlined in §194.22, §194.23 and ASME NQA-2a-1990 addenda, part 2.7. The Agency therefore 

concludes that PAPDB Version 2.00 is in compliance with §194.22, §194.23 and ASME NQA2a-1990 addenda, part 2.7 and that PAPDB Version 2.00 is approved for WIPP PA use in PA 

calculations.
12

5.0 REFERENCES

Chavez, M. J. 2006. NP 19-1 Revision 12, Software Requirements. Sandia National

Laboratories, Carlsbad, NM. ERMS #543742.

Gilkey, A. P. 2012. WIPP PA User's Manual for SDBREAD_LIB Version 4.00, Sandia

National Laboratories, Carlsbad, NM. ERMS #556534.

Kirchner, T. B., and J. Long. 2012. Report on the Migration of the WIPP PA Parameter Database 

from SQL Server to MySQL. Sandia National Laboratories, Carlsbad, NM. ERMS #557128.

Kirchner, T. B. 2011. Analysis Plan for Migration of the WIPP PA Parameter Database from

SQL Server to MySQL. Carlsbad, NM: Sandia National Laboratories. ERMS # 556953.

Kirchner, T. B. 2011a. Software QA Plan Criteria PAPDB Version 2.00 Sandia National 

Laboratories, Carlsbad, NM. ERMS #555997.

Kirchner, T. B. 2011b. The PAPDB Version 2.00 Requirements Document. Sandia

National Laboratories, Carlsbad, NM. ERMS #555998.

Kirchner, T. B. 2011c. The PAPDB Version 2.00 Verification and Validation Plan.

Sandia National Laboratories, Carlsbad, NM. ERMS #556729.

Kirchner, T. B. 2012. Design Document PAPDB Version 2.00 Sandia National Laboratories, 

Carlsbad, NM. ERMS #556730.

Kirchner, T. B. 2012a. Validation Document PAPDB Version 2.00 Sandia National Laboratories, 

Carlsbad, NM. ERMS #556001.

Kirchner, T. B. 2012b. Installation and Checkout PAPDB Version 2.00 Sandia National 

Laboratories, Carlsbad, NM. ERMS #556002.

Kirkes, G.R. and C.G. Herrick. 2006. Analysis Plan for the Modification of the Waste Shear 

Strength Parameter and Direct Brine Release Parameters, AP-131.Sandia National Laboratories, 

Carlsbad, NM. ERMS 541530.