A-1

APPENDIX A: VERIFICATION CODES

A1.0 MODFLOW2000

The MODFLOW ground water software has been developed by the USGS and has been 

continuously upgraded since the first version, MODFLOW88 was released in 1988 (McDonald 

and Harbaugh 2003). MODFLOW-2005 is the newest version currently available. Together, the 

different versions of MODFLOW have an international user base of over 1,000 users. As a 

ground water flow and transport simulator, the MODFLOW and related software is suited to the 

simulation of a wide variety of saturated zone flow and transport problems.

As part of EPA’s regulatory oversight of the Waste Isolation Pilot Plant (WIPP), located in 

southeastern New Mexico, the U.S. Department of Energy (the Department or DOE) that 

operates the WIPP repository must continue to meet the certification requirements of the EPA

and, in part, must demonstrate on an ongoing basis that Performance Assessment computer 

software is in compliance with regulations outlined in Section §194.23 – Models and Computer 

Codes. This section presents DOE’s verification and validation of MODFLOW2000.

A1.1 INTRODUCTION

MODFLOW is a computer program that numerically solves the three-dimensional ground water

flow equation for a porous medium by using a finite-difference method. MODFLOW is designed 

to be modular, in that different functionalities, such as wells, rivers, evapotranspiration, etc., can 

be added as modules to the basic ground water flow solutions. Although MODFLOW was 

designed to be easily enhanced, the design was oriented toward additions to the ground water

flow equation. Frequently, there is a need to solve additional equations; for example, transport 

equations and equations for estimating parameter values that produce the closest match between 

model-calculated heads and flows and measured values. The version of MODFLOW used by 

DOE, MODFLOW2000 (MF2K) is designed to simulate more complex boundary conditions

(WIPP 2002; Donald and Harbaugh 1988; Harbaugh et al. 2000; MODFLOW 2000). The user’s 

manual for MODFLOW 2000 (Harbaugh et al. 2000) contains an overview of the old and added 

design concepts, documents one new package, and contains input instructions for using the 

model to solve the ground water flow equation. For transient and steady-state, single-phase, 

ground water flow problems, the MODFLOW2000 software is executed with the prescribed 

boundary and initial conditions. MODFLOW was not used for the CCA.

Software Requirements (SNL NP 19-1) requires that the following seven primary documents be 

developed, reviewed, and maintained for the MODFLOW software:

Software Quality Assurance (QA) Plan

Requirements Document (RD)

Verification and Validation Plan (VVP)

User’s Manual (UM)

Design Document (DD)

Implementation Document (ID)

Validation Document (VD)
A-2

DOE reviewed the pre-existing documentation available for MODFLOW2000 from the USGS 

and found it to provide the necessary information that is usually within the RD, DD, UM, and 

VVP. Therefore, the only additional documents that were produced by DOE are the Software QA 

Plan (WIPP PA 2003c), the ID (WIPP PA 2003b), VD (WIPP PA 2003a), and the Installation 

and Check Out forms (WIPP PA 2003d). DOE notes that documentation for Version 1.6 will 

remain as the base document for any future versions of the software, with addenda for each of 

the documents defining the additional scope of the revised software. Configuration control is 

maintained through completion of Installation & Checkout (I&C) documentation for all changes 

made to MODFLOW2000 and system software and/or system hardware. In addition, Change 

Control (CC) and Software Problem Report (SPR) documents are completed, as appropriate.

The construction of newer clusters of Linux-based computers has required the testing of certain 

codes that have been previously qualified on older hardware. 

In 2003, MODFLOW2000 Version 1.6 was qualified for use on the PC-based Linux cluster

(WIPP PA 2003a). The Agency reviewed DOE’s qualification and accepted the verification of 

MODFLOW2000 Version 1.6 on the Linux platform (EPA 2004). DOE used these EPA approved 

software and hardware configurations to support CRA-2004 and PABC-2004.

The Linux-based cluster was upgraded in 2006 (new processors and other hardware) and is now 

called the “Geo-Hydro Linux Cluster” (WIPP PA 2008). This cluster is comprised of three 

different hardware groups, each with a group name: (1) eleionomae, (2) pegaeae, and 

(3) crinaeae. The computers are connected to a job control server, “tethys.sandia.gov,” which is 

not used for execution of codes. Because the hardware is new, but the software codes are 

unchanged and are not going to be recompiled, DOE only conducted regression testing to 

validate that the codes perform correctly on the new systems. For both CRA-2009 and PABC2009, DOE used MODFLOW 1.6 in conjunction with the three hardware groups associated with 

the “Geo-Hydro Linux Cluster” mentioned above. The approach, results and Agency findings 

pertaining to this upgraded hardware are discussed below.

A1.2 TEST METHODOLOGY

The DOE designed eight test cases to verify the functional requirements necessary for the 

verification/validation of the computer code for WIPP. The input files and corresponding output 

files are provided with the installation package. Listings of these files are included in 

Appendix A to the VD corresponding to the test number and test name. Validation testing 

consisted of running all test cases and checking resulting output for consistency with 

documented results. The test cases were run with the production executable (e.g., the executable 

version used for PA compliance calculations) for MF2K. The production executable was created 

on the target platform by the code sponsor and stored using CVS (e.g., Concurrent Versions 

System) version control on the target platform (CVSROOT - /h/WIPPcvs, repository - src/mf2k). 

The executable, source code and test problems were also stored in SCMS on the WIPP Compaq 

Alpha cluster (Library- MF2K, class- VER_0160).

The MF2K production executable and input and output test files were obtained from 

configuration management and placed in the test directories on the target platform. All of the 
A-3

input files were used unmodified from the source code package, except for the *.nam file, where 

the file pathnames were modified to reflect the different syntax between the Windows and Linux 

operating systems. The MF2K output listing files, *.lst, created during testing were compared to 

the output listing files obtained from the MF2K installation package, and differences were noted 

and addressed. The listing file is the primary ASCII text file created by MF2K and contains an 

input echo, solver performance information, calculated head and a budget summary. This same 

procedure was used for all the tests, with the exception of Test Case 8, the algebraic multi-grid 

(AMG) test. The intent of Test Case 8 is to verify the Linked algebraic Multi-Grid solver (LMG) 

package that was not included in the MODFLOW2000 test suite. A test identical to Test Case 1, 

BCF2SS, was chosen, except that the solver has been switched from the Strongly Implicit 

Procedure (SIP) to the LMG or linked algebraic Multi-Grid solver. The results of Test Case 8 

were compared to the results of Test Case 1.

After the code was verified, it was regression tested against the verification results (WIPP PA 

2009a, 2009b, and 2009c). The run-control for these tests was done using the csh script 

RunReadScript and the Python programs ReadScript.py and Format.py. RunReadScript was used 

to run ReadScript.py (for processing the list of files to be checked out, checked in, executed and 

compared), run Fomat.py (for formatting the output of ReadScript.py into an Word file), and 

then check the log and Word files into the repository. The specific input script and the locations 

within the CVS repository where the input script and log file can be found are presented in the 

regression test documentation (WIPP PA 2009a, 2009b, and 2009c).

The UNIX diff (e.g., difference) command was used to compare the output to original data. The 

diff command does a character-by-character comparison of two ASCII files (binary files cannot 

be compared). Any differences are reported by listing the line number in the first file, the type of 

change (“a” for addition, “c” for change, “d” for deletion), and then the line numbers in the 

second file.

The test was considered successful if the MODFLOW2000 output listing file was the same as the 

documented listing file within reasonable accuracy, and accounting for date and filename 

changes. Reasonable accuracy was defined as numerically equal, except in the last printed digit 

for numbers printed with 6 or less digits, or in the digits greater than the 6th for numbers printed 

with greater than 6 digits. Original output files are listed in the appendix, while the output files 

generated during testing were stored in CVS on the target platform and in SCMS accessible from 

the WIPP VMS Alpha cluster. The same criteria were used for all the test cases.

A1.3 TEST RESULTS

The regression testing performed on the “eleionomae” cluster shows that MODFLOW2000 

Version 1.6 is working in the same manner as the software performed on the original test 

platform.

A1.4 THE AGENCY’S CONCLUSIONS

All test results met the acceptance criteria specified in Section 5.2 of the VD (WIPP PA 2003a). 

Those differences that were present were character differences, due to the addition of the build 
A-4

date to the listing header, and in syntax differences between Windows and Linux pathnames, or 

were floating-point differences in insignificant digits. The AMG/LMG test produced very

different output listings due to solver-specific output, but the head results and ground water

budgets were the same as the PCG (Test Case 1, BCF2SS) results to within the acceptance 

criteria. Therefore, the Agency concludes that MODFLOW2000 Version 1.6 can be considered 

verified for use on the “eleionomae, pegaeae, and crinaeae” cluster of machines.

A2.0 MT3D

MT3D (Modular 3D Solute Transport Model) is a comprehensive three-dimensional numerical 

model for simulating solute transport in complex hydrogeologic settings. MT3D has a modular 

design that permits simulation of transport processes independently or jointly. MT3D is capable 

of modeling advection in complex steady-state and transient flow fields, anisotropic dispersion, 

first-order decay and production reactions, and linear and nonlinear sorption. It can also handle 

bioplume-type reactions, monad reactions, and daughter products. This enables MT3D to do 

multi-species reactions and simulate or assess natural attenuation within a contaminant plume. 

MT3D is linked with the USGS ground water flow simulator, MODFLOW, and is designed

specifically to handle advectively dominated transport problems without the need to construct 

refined models specifically for solute transport. 

MT3D was developed by the University of Alabama’s Hydrogeology Group and the verification 

of MT3D is provided in Zheng (2006) and Zheng and Wang (1999).

A3.0 REFERENCES 

Donald, M.G., and A.W. Harbaugh, A.W., 1988. A Modular Three-Dimensional FiniteDifference Ground-water Flow Model, TWI 6-A1, 588 p. U.S. Geologic Survey, Reston, 

Virginia. ERMS #522202.

EPA 2004. “Review of WIPP Performance Assessment Computer Code Migration, March 31, 

2004.” Environmental Protection Agency.

Harbaugh, A.W., E.R. Banta, M.C. Hill, and M.G. McDonald, 2000. MODFLOW-2000, The U.S. 

Geological Survey Modular Ground-Water Model – User Guide To Modularization Concepts 

And The Ground-Water Flow Process, U.S. Geological Survey Open-File Report 00-92. ERMS 

#522197.

MODFLOW 2000. The U.S. Geological Survey Modular Ground-Water Model – User Guide to 

the LINK-AMG (LMG) Package for Solving MATRIX Equations Using an Algebraic Multigrid 

Solver, MODFLOW-2000. U.S. Geological Survey Open-File Report 00-92. ERMS #52220.

WIPP PA 2002. Code Classification and Review of Pre-Existing Documentation for 

MODFLOW. Memo from Sean McKenna to Mario Chavez, Sept. 30, 2002, Sandia National 

Laboratories, Albuquerque, New Mexico. ERMS #523942.
A-5

WIPP PA 2003a. “Validation Document for MODFLOW-2000 Version 1.6 dated March 2003.” 

Sandia WIPP Central Files. ERMS#523867

WIPP PA 2003b. “Implementation Document for MODFLOW-2000 Version 1.6 dated March 

2003.” Sandia WIPP Central Files. ERMS#523868.

WIPP PA 2003a. “QA Plan for MODFLOW-2000 Version 1.6 dated March 2003.” Sandia 

WIPP Central Files. ERMS#523869.

WIPP PA 2003d. “Installation and Checkout for MODFLOW-2000 Version 1.6 dated March 5, 

2003.”

WIPP PA (Performance Assessment) 2008. “Change Control Form for MODFLOW 2000, 

Version 1.6.” Sandia National Laboratories. Sandia WIPP Central Files WPO. ERMS #550152.

WIPP PA 2009a. “Regression Testing Document for MODFLOW-2000 Version 1.6 on the 

“eleionomae” subset of the GeoHydro cluster dated February 25, 2009.” Sandia National 

Laboratories. Sandia WIPP Central Files. ERMS #550911.

WIPP PA 2009b. “Regression Testing Document for MODFLOW-2000 Version 1.6 on the 

“pegaeae” subset of the GeoHydro cluster dated February 25, 2009.” Sandia National 

Laboratories. Sandia WIPP Central Files. ERMS #550915.

WIPP PA 2009c. “Regression Testing Document for MODFLOW-2000 Version 1.6 on the 

“crinaeae” subset of the GeoHydro cluster dated February 25, 2009.” Sandia National 

Laboratories. Sandia WIPP Central Files. ERMS #550916.

Zheng, C., 2006. MT3DMS v5.2 A Modular Three-Dimensional Multispecies Transport Model 

for Simulation of Advection, Dispersion and Chemical Reactions of Contaminants in 

Groundwater Systems Supplemental User's Guide, University of Alabama, Tucsaloosa, 

Alabama.

Zheng, C. and P. Wang, 1999. MT3DMS: A Modular Three-Dimensional Multispecies Transport 

Model for Simulation of Advection, Dispersion, and Chemical Reactions of Contaminants in 

Groundwater Systems; Documentation and User's Guide, SERDP-99-1, University of Alabama, 

Tuscaloosa, Alabama.