Document ID: EPA-HQ-OAR-2012-0788-0020
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2015-01-26T05:00Z

Appendix A: Verification Codes

MODFLOW2000

The MODFLOW ground water software has been developed by the USGS and has
been continuously upgraded since the first version, MODFLOW88 was
released in 1988 (McDonald and Harbaugh 2003). MODFLOW-2005 is the
newest version currently available. Together, the different versions of
MODFLOW have an international user base of over 1,000 users. As a ground
water flow and transport simulator, the MODFLOW and related software is
suited to the simulation of a wide variety of saturated zone flow and
transport problems.

As part of EPA’s regulatory oversight of the Waste Isolation Pilot
Plant (WIPP), located in southeastern New Mexico, the U.S. Department of
Energy (the Department or DOE) that operates the WIPP repository must
continue to meet the certification requirements of the EPA and, in part,
must demonstrate on an ongoing basis that Performance Assessment
computer software is in compliance with regulations outlined in Section
§194.23 – Models and Computer Codes. This section presents DOE’s
verification and validation of MODFLOW2000.

Introduction

MODFLOW is a computer program that numerically solves the
three-dimensional ground water flow equation for a porous medium by
using a finite-difference method. MODFLOW is designed to be modular, in
that different functionalities, such as wells, rivers,
evapotranspiration, etc., can be added as modules to the basic ground
water flow solutions. Although MODFLOW was designed to be easily
enhanced, the design was oriented toward additions to the ground water
flow equation. Frequently, there is a need to solve additional
equations; for example, transport equations and equations for estimating
parameter values that produce the closest match between model-calculated
heads and flows and measured values. The version of MODFLOW used by DOE,
MODFLOW2000 (MF2K) is designed to simulate more complex boundary
conditions (WIPP 2002; Donald and Harbaugh 1988; Harbaugh et al. 2000;
MODFLOW 2000). The user’s manual for MODFLOW 2000 (Harbaugh et al.
2000) contains an overview of the old and added design concepts,
documents one new package, and contains input instructions for using the
model to solve the ground water flow equation. For transient and
steady-state, single-phase, ground water flow problems, the MODFLOW2000
software is executed with the prescribed boundary and initial
conditions. MODFLOW was not used for the CCA.

Software Requirements (SNL NP 19-1) requires that the following seven
primary documents be developed, reviewed, and maintained for the MODFLOW
software:

Software Quality Assurance (QA) Plan

Requirements Document (RD)

Verification and Validation Plan (VVP)

User’s Manual (UM)

Design Document (DD)

Implementation Document (ID)

Validation Document (VD)

DOE reviewed the pre-existing documentation available for MODFLOW2000
from the USGS and found it to provide the necessary information that is
usually within the RD, DD, UM, and VVP. Therefore, the only additional
documents that were produced by DOE are the Software QA Plan (WIPP PA
2003c), the ID (WIPP PA 2003b), VD (WIPP PA 2003a), and the Installation
and Check Out forms (WIPP PA 2003d). DOE notes that documentation for
Version 1.6 will remain as the base document for any future versions of
the software, with addenda for each of the documents defining the
additional scope of the revised software. Configuration control is
maintained through completion of Installation & Checkout (I&C)
documentation for all changes made to MODFLOW2000 and system software
and/or system hardware. In addition, Change Control (CC) and Software
Problem Report (SPR) documents are completed, as appropriate.

The construction of newer clusters of Linux-based computers has required
the testing of certain codes that have been previously qualified on
older hardware. 

In 2003, MODFLOW2000 Version 1.6 was qualified for use on the PC-based
Linux cluster (WIPP PA 2003a). The Agency reviewed DOE’s qualification
and accepted the verification of MODFLOW2000 Version 1.6 on the Linux
platform (EPA 2004). DOE used these EPA approved software and hardware
configurations to support CRA-2004 and PABC-2004.

The Linux-based cluster was upgraded in 2006 (new processors and other
hardware) and is now called the “Geo-Hydro Linux Cluster” (WIPP PA
2008). This cluster is comprised of three different hardware groups,
each with a group name:  (1) eleionomae, (2) pegaeae, and (3) crinaeae.
The computers are connected to a job control server,
“tethys.sandia.gov,” which is not used for execution of codes.
Because the hardware is new, but the software codes are unchanged and
are not going to be recompiled, DOE only conducted regression testing to
validate that the codes perform correctly on the new systems. For both
CRA-2009 and PABC-2009, DOE used MODFLOW 1.6 in conjunction with the
three hardware groups associated with the “Geo-Hydro Linux Cluster”
mentioned above. The approach, results and Agency findings pertaining to
this upgraded hardware are discussed below.

Test Methodology

The DOE designed eight test cases to verify the functional requirements
necessary for the verification/validation of the computer code for WIPP.
The input files and corresponding output files are provided with the
installation package. Listings of these files are included in
Appendix A to the VD corresponding to the test number and test name.
Validation testing consisted of running all test cases and checking
resulting output for consistency with documented results. The test cases
were run with the production executable (e.g., the executable version
used for PA compliance calculations) for MF2K. The production executable
was created on the target platform by the code sponsor and stored using
CVS (e.g., Concurrent Versions System) version control on the target
platform (CVSROOT - /h/WIPPcvs, repository - src/mf2k). The executable,
source code and test problems were also stored in SCMS on the WIPP
Compaq Alpha cluster (Library- MF2K, class- VER_0160).

The MF2K production executable and input and output test files were
obtained from configuration management and placed in the test
directories on the target platform. All of the input files were used
unmodified from the source code package, except for the *.nam file,
where the file pathnames were modified to reflect the different syntax
between the Windows and Linux operating systems. The MF2K output listing
files, *.lst, created during testing were compared to the output listing
files obtained from the MF2K installation package, and differences were
noted and addressed. The listing file is the primary ASCII text file
created by MF2K and contains an input echo, solver performance
information, calculated head and a budget summary. This same procedure
was used for all the tests, with the exception of Test Case 8, the
algebraic multi-grid (AMG) test. The intent of Test Case 8 is to verify
the Linked algebraic Multi-Grid solver (LMG) package that was not
included in the MODFLOW2000 test suite. A test identical to Test Case 1,
BCF2SS, was chosen, except that the solver has been switched from the
Strongly Implicit Procedure (SIP) to the LMG or linked algebraic
Multi-Grid solver. The results of Test Case 8 were compared to the
results of Test Case 1.

After the code was verified, it was regression tested against the
verification results (WIPP PA 2009a, 2009b, and 2009c). The run-control
for these tests was done using the csh script RunReadScript and the
Python programs ReadScript.py and Format.py. RunReadScript was used to
run ReadScript.py (for processing the list of files to be checked out,
checked in, executed and compared), run Fomat.py (for formatting the
output of ReadScript.py into an Word file), and then check the log and
Word files into the repository. The specific input script and the
locations within the CVS repository where the input script and log file
can be found are presented in the regression test documentation (WIPP PA
2009a, 2009b, and 2009c).

The UNIX diff (e.g., difference) command was used to compare the output
to original data. The diff command does a character-by-character
comparison of two ASCII files (binary files cannot be compared). Any
differences are reported by listing the line number in the first file,
the type of change (“a” for addition, “c” for change, “d”
for deletion), and then the line numbers in the second file.

The test was considered successful if the MODFLOW2000 output listing
file was the same as the documented listing file within reasonable
accuracy, and accounting for date and filename changes. Reasonable
accuracy was defined as numerically equal, except in the last printed
digit for numbers printed with 6 or less digits, or in the digits
greater than the 6th for numbers printed with greater than 6 digits.
Original output files are listed in the appendix, while the output files
generated during testing were stored in CVS on the target platform and
in SCMS accessible from the WIPP VMS Alpha cluster. The same criteria
were used for all the test cases.

Test Results

The regression testing performed on the “eleionomae” cluster shows
that MODFLOW2000 Version 1.6 is working in the same manner as the
software performed on the original test platform.

The Agency’s Conclusions

All test results met the acceptance criteria specified in Section 5.2 of
the VD (WIPP PA 2003a). Those differences that were present were
character differences, due to the addition of the build date to the
listing header, and in syntax differences between Windows and Linux
pathnames, or were floating-point differences in insignificant digits.
The AMG/LMG test produced very different output listings due to
solver-specific output, but the head results and ground water budgets
were the same as the PCG (Test Case 1, BCF2SS) results to within the
acceptance criteria. Therefore, the Agency concludes that MODFLOW2000
Version 1.6 can be considered verified for use on the “eleionomae,
pegaeae, and crinaeae” cluster of machines.

MT3D

MT3D (Modular 3D Solute Transport Model) is a comprehensive
three-dimensional numerical model for simulating solute transport in
complex hydrogeologic settings. MT3D has a modular design that permits
simulation of transport processes independently or jointly. MT3D is
capable of modeling advection in complex steady-state and transient flow
fields, anisotropic dispersion, first-order decay and production
reactions, and linear and nonlinear sorption. It can also handle
bioplume-type reactions, monad reactions, and daughter products. This
enables MT3D to do multi-species reactions and simulate or assess
natural attenuation within a contaminant plume. MT3D is linked with the
USGS ground water flow simulator, MODFLOW, and is designed specifically
to handle advectively dominated transport problems without the need to
construct refined models specifically for solute transport. 

MT3D was developed by the University of Alabama’s Hydrogeology Group
and the verification of MT3D is provided in Zheng (2006) and Zheng and
Wang (1999).

References 

Donald, M.G., and A.W. Harbaugh, A.W., 1988. A Modular Three-Dimensional
Finite-Difference Ground-water Flow Model, TWI 6-A1, 588 p. U.S.
Geologic Survey, Reston, Virginia. ERMS #522202.

EPA 2004. “Review of WIPP Performance Assessment Computer Code
Migration, March 31, 2004.”  Environmental Protection Agency.

Harbaugh, A.W., E.R. Banta, M.C. Hill, and M.G. McDonald, 2000.
MODFLOW-2000, The U.S. Geological Survey Modular Ground-Water Model –
User Guide To Modularization Concepts And The Ground-Water Flow Process,
U.S. Geological Survey Open-File Report 00-92. ERMS #522197.

MODFLOW 2000. The U.S. Geological Survey Modular Ground-Water Model –
User Guide to the LINK-AMG (LMG) Package for Solving MATRIX Equations
Using an Algebraic Multigrid Solver, MODFLOW-2000. U.S. Geological
Survey Open-File Report 00-92. ERMS #52220.

WIPP PA 2002. Code Classification and Review of Pre-Existing
Documentation for MODFLOW. Memo from Sean McKenna to Mario Chavez, Sept.
30, 2002, Sandia National Laboratories, Albuquerque, New Mexico. ERMS
#523942.

WIPP PA 2003a. “Validation Document for MODFLOW-2000 Version 1.6 dated
March 2003.”  Sandia WIPP Central Files. ERMS#523867

WIPP PA 2003b. “Implementation Document for MODFLOW-2000 Version 1.6
dated March 2003.”  Sandia WIPP Central Files. ERMS#523868.

WIPP PA 2003a. “QA Plan for MODFLOW-2000 Version 1.6 dated March
2003.”  Sandia WIPP Central Files. ERMS#523869.

WIPP PA 2003d. “Installation and Checkout for MODFLOW-2000 Version 1.6
dated March 5, 2003.”

WIPP PA (Performance Assessment) 2008. “Change Control Form for
MODFLOW 2000, Version 1.6.”  Sandia National Laboratories. Sandia WIPP
Central Files WPO. ERMS #550152.

WIPP PA 2009a. “Regression Testing Document for MODFLOW-2000 Version
1.6 on the “eleionomae” subset of the GeoHydro cluster dated
February 25, 2009.”  Sandia National Laboratories. Sandia WIPP Central
Files. ERMS #550911.

WIPP PA 2009b. “Regression Testing Document for MODFLOW-2000 Version
1.6 on the “pegaeae” subset of the GeoHydro cluster dated February
25, 2009.”  Sandia National Laboratories. Sandia WIPP Central Files.
ERMS #550915.

WIPP PA 2009c. “Regression Testing Document for MODFLOW-2000 Version
1.6 on the “crinaeae” subset of the GeoHydro cluster dated February
25, 2009.”  Sandia National Laboratories. Sandia WIPP Central Files.
ERMS #550916.

Zheng, C., 2006. MT3DMS v5.2 A Modular Three-Dimensional Multispecies
Transport Model for Simulation of Advection, Dispersion and Chemical
Reactions of Contaminants in Groundwater Systems Supplemental User's
Guide, University of Alabama, Tucsaloosa, Alabama.

Zheng, C. and P. Wang, 1999. MT3DMS: A Modular Three-Dimensional
Multispecies Transport Model for Simulation of Advection, Dispersion,
and Chemical Reactions of Contaminants in Groundwater Systems;
Documentation and User's Guide, SERDP-99-1, University of Alabama,
Tuscaloosa, Alabama. 

A-  PAGE   \* MERGEFORMAT  1