-   
-   
-   
-   
-   
-   

![](media/image2.wmf){width="1.0833333333333333in"
height="1.0291666666666666in"}

> **UNITED STATES ENVIRONMENTAL PROTECTION AGENCY**

**WASHINGTON D.C. 20460**

> **OFFICE OF THE ADMINISTRATOR**
>
> **SCIENCE ADVISORY BOARD**

August 7, 2008

EPA-SAB-08-010

The Honorable Stephen L. Johnson

Administrator

U.S. Environmental Protection Agency

1200 Pennsylvania Avenue, N.W.

Washington, D.C. 20460

> Subject: Review of Draft ***"Multi-Agency Radiation Survey and
> Assessment of Materials and Equipment (MARSAME) Manual"***

Dear Administrator Johnson:

At the request of EPA's Office of Radiation and Indoor Air, the Science
Advisory Board (SAB) Radiation Advisory Committee (RAC), augmented with
additional experts (referred herein as the Panel or the SAB MARSAME
Review Panel), has completed its review of the ***"Multi-Agency
Radiation Survey and Assessment of Materials and Equipment (MARSAME)
Manual,"(December 2006 Draft).*** This draft manual was prepared by a
multi-agency work group with participation from the U.S. Department of
Energy, the U.S. Nuclear Regulatory Commission, the U.S. Department of
Defense, and the U.S. Environmental Protection Agency (U.S. EPA). The
multi-agency work group has been active since 1995, for some periods
with representation from additional federal agencies, to prepare a
series of radiological guidance documents, of which this is the third.
The preceding documents are entitled ***"Multi-Agency Radiation Survey
and Site Investigation Manual (MARSSIM)"*** and ***"Multi-Agency
Radiological Laboratory Analytical Protocols (MARLAP) Manual."*** These
manuals were previously reviewed by the RAC, augmented with additional
experts.

The MARSAME manual complements MARSSIM (a surface soils radiation survey
manual) by providing a process for surveying potentially radioactive
material and equipment (M&E). It is a detailed document that provides
guidance to determine whether M&E are sufficiently free of radionuclide
contamination to be admitted to or removed from a site. Its chapters
address the components of a survey plan: initial assessment, input
needed for decision making, survey design, survey implementation, and
reaching a disposition decision. The manual begins with a road map to
help the user navigate the manual, includes a chapter with illustrative
examples, and collects pertinent information in seven appendices. Much
of its presentation is based on the contents of MARSSIM and MARLAP
because M&E surveys often are related to site investigations and utilize
laboratory analyses; however, an M&E survey may stand alone.

The SAB found the MARSAME manual to be an admirable cooperative and
competently written effort by staff from the several agencies that
provides guidance in an important endeavor. The Panel expects the manual
to be as widely applied as the two earlier radiological guidance manuals
and to contribute significantly to radiation protection for the US
population. To assist this endeavor, the Panel presents 37
Recommendations and a Statistical Analysis Appendix in the enclosed
review.

The main recommendations are:

-   To facilitate application of MARSAME by important users such as
    project managers and site cleanup specialists who are not radiation
    protection professionals, support the Manual by holding training
    courses.

-   To improve ease in reading and applying MARSAME, combine in a
    separate chapter the detailed guidance for hypothesis testing,
    experimental design, and statistical analysis that is now dispersed
    throughout the Manual.

-   To clarify application of MARSAME guidance, improve the illustrative
    examples (now mislabeled 'case studies') by replacing them with real
    case studies, or enhance the illustrative examples with additional
    information based on real situations.

-   To permit realistic design of M&E surveys in response to required
    action levels, include in MARSAME the pertinent regulations and
    guidance, and provide an updating mechanism.

-   To avoid serious errors in delineating the radioactive contaminants
    of M&E, provide in MARSAME the same detailed discussion of
    radionuclides that are categorized as (1) removable from the surface
    and (2) distributed throughout the material volume as is currently
    devoted to the category of surface contamination that may be either
    fixed or removable.

-   To expand the range of alternative M&E surveys available in MARSAME
    -- from application of the full content of the Manual to 'no further
    action needed,' include descriptions of iterative release efforts
    that may include decontamination efforts and storage for radioactive
    decay.

Other recommendations concern additional refinements and improvements in
content and presentation, including the concept that MARSAME be updated
periodically.

The SAB appreciates the opportunity to review this draft manual and
hopes that the recommendations provided will enable EPA and cooperating
agencies to issue effective guidance for radiological surveys of
material and equipment.

The revised MARSAME Manual will be a useful document for EPA and other
Federal and State agencies in providing guidance to control transfer of
material and equipment that may be contaminated with radionuclides. We
look forward to the Agency's response.

> Sincerely,

***/Signed/ /Signed/***

Dr. M. Granger Morgan Dr. Bernd Kahn

Chair, Science Advisory Board Chair, Radiation Advisory Committee

**NOTICE**

This report has been written as part of the activities of the EPA
Science Advisory Board (SAB), a public advisory group providing
extramural scientific information and advice to the Administrator and
other officials of the Environmental Protection Agency. The SAB is
structured to provide balanced, expert assessment of scientific matters
related to problems facing the Agency. This report has not been reviewed
for approval by the Agency and, hence, the contents of this advisory do
not necessarily represent the views and policies of the Environmental
Protection Agency, nor of other agencies in the Executive Branch of the
Federal government, nor does mention of trade names of commercial
products constitute a recommendation for use. Reports and advisories of
the SAB are posted on the EPA website at <http://www.epa.gov/sab>.

**U.S. Environmental Protection Agency**

**Science Advisory Board**

**Radiation Advisory Committee (RAC) Augmented for Review of the**

**Multi-Agency Radiation Survey and Assessment of Materials and
Equipment (MARSAME) Manual**

**CHAIR**:

**Dr. Bernd Kahn**, Professor Emeritus, Nuclear and Radiological
Engineering Program, and Director, Environmental Radiation Center, GTRI,
Georgia Institute of Technology, Atlanta, GA

**PAST CHAIR**:

**Dr. Jill Lipoti**, Director, Division of Environmental Safety and
Health, New Jersey Department of Environmental Protection, Trenton, NJ

**RAC MEMBERS**:

**Dr. Thomas B. Borak**, Professor, Department of Environmental and
Radiological Health Sciences, Colorado State University, Fort Collins,
CO

**Dr. Antone L. Brooks,** Professor, Radiation Toxicology, Washington
State University Tri-Cities, Richland, WA

**Dr. Faith G. Davis**, Senior Associate Dean, Professor of
Epidemiology, Division of Epidemiology and Biostatistics, School of
Public Health, University of Illinois at Chicago, Chicago, IL

**Dr. Brian Dodd,** Consultant, Las Vegas, NV

**Dr. Shirley A. Fry,** Consultant, Indianapolis, IN

**Dr. William C. Griffith,** Associate Director, Institute for Risk
Analysis and Risk Communication, Department of Environmental and
Occupational Health Sciences, University of Washington, Seattle, WA

**Dr. Jonathan M. Links**, Professor, Department of Environmental Health
Sciences, Bloomberg School of Public Health, Johns Hopkins University,
Baltimore, MD

**Mr. Bruce A. Napier,** Staff Scientist, Radiological Science &
Engineering Group, Pacific Northwest National Laboratory, Richland,
WA[^1]

**Dr. Daniel O. Stram**, Professor, Department of Preventive Medicine,
Division of Biostatistics and Genetic Epidemiology, Keck School of
Medicine, University of Southern California, Los Angeles, CA

**Dr. Richard J. Vetter,** Head, Radiation Safety Program, Mayo Clinic,
Rochester, MN

**CONSULTANTS**:

**Mr. Bruce W. Church**, President, BWC Enterprises, Inc., Hurricane, UT

**Mr. Kenneth Duvall**, Environmental Scientist/Consultant, Washington,
D.C.

**Dr. Janet A. Johnson**, Consultant, Carbondale, CO

**Dr. Paul J. Merges**, President, Environment & Radiation Specialists,
Inc., Loudonville, N.Y.

**SCIENCE ADVISORY BOARD STAFF**

**Dr. K. Jack Kooyoomjian**, Designated Federal Officer, 1200
Pennsylvania Avenue, NW, Washington, DC, 20460-0001, Phone:
202-343-9984, Fax: 202-233-0643, or 0645
([kooyoomjian.jack@epa.gov]{.underline}) Messenger/Physical Delivery
Address: 1025 F Street, NW, Room 3606, Mail Code 1400F

**U.S. Environmental Protection Agency**

**Science Advisory Board**

**CHAIR**

**Dr. M. Granger Morgan**, Lord Chair Professor in Engineering,
Department of Engineering and Public Policy, Carnegie Mellon University,
Pittsburgh, PA

**MEMBERS**

**Dr. Gregory Biddinger**, Coordinator, Natural Land Management
Programs, Toxicicology and Environmental Sciences, ExxonMobil Biomedical
Sciences, Inc, Houston, TX

**Dr. Thomas Burke**, Professor, Department of Health Policy and
Management, Johns Hopkins Bloomberg School of Public Health, Johns
Hopkins University, Baltimore, MD

**Dr. James Bus**, Director of External Technology, Toxicology and
Environmental Research and Consulting, The Dow Chemical Company,
Midland, MI

**Dr. Deborah Cory-Slechta**, Professor, Department of Environmental
Medicine, School of Medicine and Dentistry, University of Rochester,
Rochester, NY

**Dr. Maureen L. Cropper**, Professor, Department of Economics,
University of Maryland, College Park, MD

**Dr. Virginia Dale**, Corporate Fellow, Environmental Sciences
Division, Oak Ridge National Laboratory, Oak Ridge, TN

**Dr. Kenneth Dickson**, Regents Professor, Department of Biological
Sciences, University of North Texas, Aubrey, TX

**Dr. David A. Dzombak**, Walter J. Blenko Sr. Professor of
Environmental Engineering, Department of Civil and Environmental
Engineering, College of Engineering, Carnegie Mellon University,
Pittsburgh, PA

**Dr. Baruch Fischhoff**, Howard Heinz University Professor, Department
of Social and Decision Sciences, Department of Engineering and Public
Policy, Carnegie Mellon University, Pittsburgh, PA

**Dr. James Galloway**, Professor, Department of Environmental Sciences,
University of Virginia, Charlottesville, VA

**Dr. James K. Hammitt**, Professor, Center for Risk Analysis, Harvard
University, Boston, MA

**Dr. Rogene Henderson**, Senior Scientist Emeritus, Lovelace
Respiratory Research Institute, Albuquerque, NM

**Dr. James H. Johnson**, Professor and Dean, College of Engineering,
Architecture & Computer Sciences, Howard University, Washington, DC

**Dr. Bernd Kahn**, Professor Emeritus, Nuclear and Radiological
Engineering Program, and Director, Environmental Radiation Center, GTRI,
Georgia Institute of Technology, Atlanta, GA

**Dr. Agnes Kane**, Professor and Chair, Department of Pathology and
Laboratory Medicine, Brown University, Providence, RI

**Dr. Meryl Karol**, Professor Emerita, Graduate School of Public
Health, University of Pittsburgh, Pittsburgh, PA

**Dr. Catherine Kling**, Professor, Department of Economics, Iowa State
University, Ames, IA

**Dr. George Lambert**, Associate Professor of Pediatrics, Director,
Center for Childhood Neurotoxicology, Robert Wood Johnson Medical
School-UMDNJ, Belle Mead, NJ

**Dr. Jill Lipoti**, Director, Division of Environmental Safety and
Health, New Jersey Department of Environmental Protection, Trenton, NJ

**Dr. Michael J. McFarland**, Associate Professor, Department of Civil
and Environmental Engineering, Utah State University, Logan, UT

**Dr. Judith L. Meyer**, Distinguished Research Professor Emeritus,
Institute of Ecology, University of Georgia, Athens, GA

**Dr. Jana Milford**, Associate Professor, Department of Mechanical
Engineering, University of Colorado, Boulder, CO

**Dr. Rebecca Parkin**, Professor and Associate Dean, Environmental and
Occupational Health, School of Public Health and Health Services, The
George Washington University Medical Center, Washington, DC

**Mr. David Rejeski**, Director, Foresight and Governance Project,
Woodrow Wilson International Center for Scholars, Washington, DC

**Dr. Stephen M. Roberts**, Professor, Department of Physiological
Sciences, Director, Center for Environmental and Human Toxicology,
University of Florida, Gainesville, FL

**Dr. Joan B. Rose**, Professor and Homer Nowlin Chair for Water
Research, Department of Fisheries and Wildlife, Michigan State
University, East Lansing, MI

**Dr. James Sanders**, Director and Professor, Skidaway Institute of
Oceanography, Savannah, GA

**Dr. Jerald Schnoor**, Allen S. Henry Chair Professor, Department of
Civil and Environmental Engineering, Co-Director, Center for Global and
Regional Environmental Research, University of Iowa, Iowa City, IA

**Dr. Kathleen Segerson**, Professor, Department of Economics,
University of Connecticut, Storrs, CT

**Dr. Kristin Shrader-Frechette**, O\'Neil Professor of Philosophy,
Department of Biological Sciences and Philosophy Department, University
of Notre Dame, Notre Dame, IN

**Dr. V. Kerry Smith**, W.P. Carey Professor of Economics , Department
of Economics , W.P Carey School of Business, Arizona State University,
Tempe, AZ

**Dr. Deborah Swackhamer**, Interim Director and Professor, Institute on
the Environment, University of Minnesota, St. Paul, MN

**Dr. Thomas L. Theis**, Director, Institute for Environmental Science
and Policy, University of Illinois at Chicago, Chicago, IL

**Dr. Valerie Thomas**, Anderson Interface Associate Professor, School
of Industrial and Systems Engineering, Georgia Institute of Technology,
Atlanta, GA

**Dr. Barton H. (Buzz) Thompson, Jr.**, Robert E. Paradise Professor of
Natural Resources Law at the Stanford Law School and Director, Woods
Institute for the Environment Director, Stanford University, Stanford,
CA

**Dr. Robert Twiss**, Professor Emeritus, University of
California-Berkeley, Ross, CA

**Dr. Lauren Zeise**, Chief, Reproductive and Cancer Hazard Assessment
Branch, Office of Environmental Health Hazard Assessment, California
Environmental Protection Agency, Oakland, CA

**SCIENCE ADVISORY BOARD STAFF**

**Mr. Thomas Miller**, Designated Federal Officer, 1200 Pennsylvania
Avenue, NW

1400F, Washington, DC, Phone: 202-343-9982, Fax: 202-233-0643

**TABLE OF CONTENTS**

1\.

[EXECUTIVE SUMMARY 1](#executive-summary)

2\. [INTRODUCTION 4](#introduction)

[2.1]{.smallcaps}
[[Background]{.smallcaps}](#__RefHeading___Toc143507193) [4]{.smallcaps}

[2.2]{.smallcaps} [[Review Process and
Acknowledgement]{.smallcaps}](#__RefHeading___Toc143507193)
[5]{.smallcaps}

[2.3 EPA Charge to the Panel]{.smallcaps} [6]{.smallcaps}

3\. [RESPONSE TO THE STATISTICS ELEMENTS OF THE CHARGE QUESTIONS
.........................
8](#response-to-charge-question-1-providing-an-approach-for-planning-conducting-evaluating-and-documenting-environmental-radiological-surveys-to-determine-the-appropriate-disposition-for-materials-and-equipment)

[4. RESPONSE TO CHARGE QUESTION 1: PROVIDING AN APPROACH FOR PLANNING,
CONDUCTING, EVALUATING AND DOCUMENTING ENVIRONMENTAL RADIOLOGICAL
SURVEYS TO DETERMINE THE APPROPRIATE DISPOSITION FOR MATERIALS AND
EQUIPMENT](#response-to-charge-question-1-providing-an-approach-for-planning-conducting-evaluating-and-documenting-environmental-radiological-surveys-to-determine-the-appropriate-disposition-for-materials-and-equipment)
9

[4.1]{.smallcaps} [[Charge Question \#
1]{.smallcaps}](#__RefHeading___Toc143507193) [9]{.smallcaps}

[4.2]{.smallcaps} [[Charge Question \#
1a]{.smallcaps}](#__RefHeading___Toc143507193) [10]{.smallcaps}

[4.3]{.smallcaps} [[Charge Question \#
1b]{.smallcaps}](#__RefHeading___Toc143507196) [11]{.smallcaps}

[4.4]{.smallcaps} [[Charge Question \#
1c]{.smallcaps}](#__RefHeading___Toc143507201) [11]{.smallcaps}

[4.5]{.smallcaps} [[Charge Question \#
1d]{.smallcaps}](#__RefHeading___Toc143507201) [12]{.smallcaps}

5\. [RESPONSE TO CHARGE QUESTION 2: COMMENTS ON THE STATISTICAL
METHODOLOGY CONSIDERED IN
MARSAME](#response-to-charge-question-2-comments-on-the-statistical-methodology-considered-in-marsame)
14

[5.1]{.smallcaps} [[Charge Question \#
2]{.smallcaps}](#__RefHeading___Toc143507193) [14]{.smallcaps}

[5.2]{.smallcaps} [[Charge Question \#
2a]{.smallcaps}](#__RefHeading___Toc143507193) [14]{.smallcaps}

[5.3]{.smallcaps} [[Charge Question \# 2b]{.smallcaps}
1](#__RefHeading___Toc143507196)[5]{.smallcaps}

[5.4]{.smallcaps} [[Charge Question
#2c]{.smallcaps}](#__RefHeading___Toc143507201) [18]{.smallcaps}

6\. [RESPONSE TO CHARGE QUESTION 3: RECOMMENDATIONS PERTAINING TO THE
MARSAME ROADMAP AND
APPENDICES](#response-to-charge-question-2-comments-on-the-statistical-methodology-considered-in-marsame)
19

7\. [RECOMMENDATIONS BEYOND THE
CHARGE](#response-to-charge-question-3-recommendations-pertaining-to-the-marsame-roadmap-and-appendices)
22

REFERENCES 23

[APPENDIX A -- STATISTICAL ANALYSIS -- AN INTRODUCTION TO EXPERIMENTAL
DESIGN AND HYPOTHESIS TESTING WITH SPECIFIC COMMENTS ON
STATISTICS](#refheading___toc143507226) 26

[A-1 An Introduction to Experimental Design and Hypothesis
Testing]{.smallcaps} [26]{.smallcaps}

[A-2 Specific Comments]{.smallcaps} [29]{.smallcaps}

[APPENDIX B --ACRONYMS AND ABBREVIATIONS](#__RefHeading___Toc143507226)

38

[APPENDIX C -- TYPOS AND CORRECTIONS](#refheading___toc143507226) 40

**TEXT FIGURES**

FIGURE 1 -- Suggested Changes for Figure 6.3 from MARSAME Manual for
Interpretation of Survey Results for Scan-Only and In Situ Surveys 16

FIGURE 2 -- Suggested Changes for Figure 6.4 from MARSAME Manual for
Interpretation of Results for MARSSIM-Type Surveys 17

FIGURE 3 -- The MARSAME Process 20

**APPENDIX A FIGURES**

FIGURE A-1 SCENARIO A 30

FIGURE A-2 SCENARIO B 30

FIGURE A-3 Comparison of S~c~ 34

FIGURE A-4 Comparison of S~c~ for longer background counting period 34

FIGURE A-5 S~c~ for a briefer background counting period 35

FIGURE A-6 1- as a function of % excess count above background 35

# EXECUTIVE SUMMARY

The Radiation Advisory Committee (RAC) of the Science Advisory Board
(SAB), augmented for this review (also hereinafter referred to as the
Panel, or the SAB MARSAME Review Panel) has completed its review of the
Agency's draft document entitled ***"Multi-Agency Radiation Survey and
Assessment of Materials and Equipment (MARSAME) Manual,"*** Draft Report
for Comment, December 2006 (U.S. EPA, 2006; see also the MARSAME Hotlink
at [http://www.marsame.org](http://www.marsame.org/)). The MARSAME
manual presents a framework for planning, implementing, and assessing
radiological surveys of material and equipment (M&E). MARSAME
supplements the Multi-Agency Radiation Survey and Site Investigation
Manual (MARSSIM; see also the MARSSIM Hotlink at
<http://epa.gov/radiation/marssim/index.html>), and refers to
information provided in the Multi-Agency Radiological Laboratory
Analytical Protocols (MARLAP) Manual. The MARLAP Hotlink is
<http://epa.gov/radiation/marlap/index.html>.

All manuals were prepared collaboratively by a multi-agency work group
comprised of professionals from several pertinent Federal agencies. The
three documents, taken together, describe radiological survey programs
in great detail and address recommendations to "technical audiences
having knowledge of health physics and statistics" for performing such
surveys. The manuals are designed to enable effective comparisons of
survey measurements of radionuclide concentrations to regulations or
guides for accepting or rejecting approval of a program or process.
Vocabulary and techniques in MARSAME are carried forward from MARSSIM
and MARLAP.

The MARSAME manual complements MARSSIM (a surface-soil radiation survey
manual known as the Multi-Agency Survey and Site Investigation Manual)
by providing a process for surveying potentially radioactive M&E that
may be in nature, commerce, or use when considered for receipt or
disposition. It presents an overview of the various aspects of initial
assessment, decision inputs, survey design, survey implementation, and
assessment of results. Important activities such as hypothesis testing
and statistical analysis of measurement reliability are described in
considerable detail. A number of illustrative examples, incorrectly
termed "case studies," are presented. A road map assists the reader in
moving among chapters. Useful information is collected in appendices.

This review of the MARSAME Manual by the SAB was requested by the EPA
Office of Radiation and Indoor Air (ORIA). It is based on reading the
*MARSAME Draft Report for Comment (December 2006),* presentations by
MARSAME multi-agency work group members at the meeting on October
29--31, 2007, and discussions in a series of teleconference meetings
held on October 9, 2007, December 21, 2007, and March 10, 2008. A
quality review of the Panel's April 24, 2008 draft report was conducted
by the chartered SAB on May 29, 2008 in a public teleconference.

The Panel recognizes the magnitude of the effort by the multi-agency
work group and the value of its product. The Panel recommends
modifications to only a small fraction of this product. Panel
recommendations can be summarized in the following broad categories:

-   MARSAME guidance is suitable for experienced radiation protection
    and surveillance staff, but supporting courses should be presented
    for involved managers and professionals with limited knowledge of
    health physics or statistics to give them the needed special
    training and information that they would otherwise have to
    assimilate by searching through the lengthy MARSSIM and MARLAP
    documents. (1-3, 3-2, 3-4, C-4)[^2]

-   Specialized guidance for applying statistical tools for data
    analysis, experimental design, and hypothesis testing should be
    separated from the otherwise pervasively non-quantitative guidance
    for the convenience of the general audience and for acceptance by
    specialists. This guidance should be in a separate chapter, enhanced
    in accord with comments in the Appendix to this review. (1b-3, 1c-1,
    2a-1, 2a-2, 2c-1, 3-6)^2^

-   The incorrectly entitled 'case studies' should be labeled as
    'illustrative examples' and their contents should be enhanced to
    assure realism. (1d-1, 1d-2, 1d-3, 1d-4, 2c-2, 2c-3)^2^

-   Known regulations and guidance for meeting M&E action levels in
    MARSAME should be tabulated or cited by reference, with a mechanism
    for updating the references. (1b-1, 3-5)^2^

-   As much consideration should be given to surveys for radioactive
    contamination that is removable from the surface and that is
    dispersed throughout the material volume as is given currently to
    surface contamination (fixed plus removable) in order to distinguish
    among the three categories for radiation protection. (1b-2, 2b-3,
    2b-4)^2^

-   The various alternatives for M&E surveys should be described in
    sufficient detail to provide a wide choice of options, from no
    further action needed through minor survey efforts to a major survey
    that applies the full contents of the MARSAME manual. The options
    should include iterative M&E release efforts such as decontamination
    or storage for decay. (1-1, 1-2, 1c-2, C-3)^2^

-   Other recommendations are intended to improve the usefulness of
    various portions of the MARSAME manual, including updating it
    periodically (1a-1, 1a-2, 1a-3, 2a-3, 2b-1, 2b-2, 3-1, 3-3, 3-7,
    3-8, C-1, C-2)^2^

The multi-agency work group clearly has devoted considerable effort to
describing the statistical tools. This is important because acceptance
of survey measurements depends on their reliability near the action
level (AL). Meeting this requirement can only be demonstrated in a
statistical framework; for example, the discrimination level (DL) must
be below the AL in Scenario A, where the DL is defined to the
satisfaction of the surveyor and the regulator in terms of the values
for allowable type I error α and the allowable type II error β.

Because of the importance of clarity in the mathematical support
structure, a sub-group of the Panel has prepared a guide to those topics
in MARSAME that is collected in Appendix A to this review. This guide is
devoted to matters such as survey design, the gray region, the DL, the
test significance levels α and β, and hypothesis testing for Scenario A
and Scenario B. The guide is intended to present to the multi-agency
work group the Panel's view of (1) making this approach readily
accessible to persons only generally familiar with statistical analysis,
and (2) gaining acceptance by those who are knowledgeable on this topic.

# 2. INTRODUCTION

## 2.1 Background 

The MARSAME Manual (U.S. EPA, 2006b) was designed to guide a radiation
protection professional through all aspects of radiological surveys of
M&E prior to intended receipt or appropriate disposition. It is written
sufficiently broadly to pertain to all types of M&E. Cited as examples
are metals, concrete, tools, trash, equipment, furniture, containers of
material, and piping, among others. Release or interdiction, i.e.,
acceptance or rejection of M&E transfer, are the alternate outcomes of
the survey.

The draft document for review was prepared collaboratively by staff
working together from the following Federal agencies: the U.S.
Environmental Protection Agency (US EPA), the U.S. Nuclear Regulatory
Commission (US NRC), the U.S. Department of Energy (US DOE), and the
U.S. Department of Defense (US DoD). It is part of a continuing and
technically significant effort that began with writing MARSSIM (U.S.
EPA./SAB,1997; U.S. EPA, 2000 and 2001), continued with MARLAP (U.S.
EPA./SAB, 2003a.; U.S. EPA, 2004), and anticipates preparation of at
least one other manual after MARSAME for sub-surface radiation surveys
and characterization. The methodology and associated vocabulary in
MARSAME follow those of the preceding manuals, although a few aspects of
MARSAME are distinct. Notably, MARSAME may be connected to MARSSIM and
MARLAP as part of a site survey, or stand by itself in considering the
transfer of M&E to or from a site.

Survey guidance in the MARSAME manual and its predecessors is based on
the Data Quality Objectives (DQO) process to design the best survey with
regard to disposition option, action level (AL), and M&E type. The Data
Life Cycle (DLC) supports DQO by carrying suitable information through
the planning, implementation, assessment, and decision stages of the
program. The data are collected, evaluated, and applied in terms of
Measurement Quality Objectives (MQO) established with statistical
concepts of data uncertainty and Minimum Quantifiable Concentrations
(MQC). The sensitivity of measurements is defined in terms of the
discrimination limit (DL), which is is attained by selecting suitable
radionuclide detectors and conditions of sampling and measurement. The
measurement results must be acceptable relative to action levels and
significance levels specified in regulations or guidance.

The MARSAME document is structured as follows, shown with the relevant
charge question (CQ) number (See Section 2.3 for the charge questions):

Acronyms and Abbreviations

Symbols, Nomenclature, and Notations

Conversion factors

Road Map (CQ 3)

Chapter 1, Introduction and overview (CQ 1)

Chapter 2, Initial assessment of M&E (CQ 1a)

Chapter 3, Identify inputs for the decision (CQ 1b)

Chapter 4, Survey design (CQ 1c)

Chapter 5, Implementation of disposition surveys (CQ 2a)

Chapter 6, Assess the results of the disposition survey (CQ 2b)

Chapter 7, Case studies (CQ 1d and 2c)

7 Appendices (CQ 3)

References

Glossary

Response to the charge questions was the primary purpose of the Panel
and is addressed first. The Panel also considered a few related topics,
commented in detail on the MARSAME discussion of statistical and
operational aspects, and suggested minor corrections.

## 2.2 Review Process and Acknowledgement 

The U.S. EPA's Office of Radiation and Indoor Air (ORIA), on behalf of
the Federal Agencies participating in the development of the draft
MARSAME Manual, requested the SAB to provide advice on the draft
document entitled *"Multi-Agency Radiation Survey and Assessment of
Materials and Equipment (MARSAME) Manual,"* *Draft Report for Comment,
December 2006* (U.S. EPA. 2006b.; also numbered as NUREG-1575, Supp. 1;
EPA 402-R-06-002; and DOE/EH-707). MARSAME is a supplement to the
*"Multi-Agency Radiation Survey and Site Investigation Manual"*
(MARSSIM; U.S. EPA, 2000 and 2001; also numbered as NUREG-1575, rev. 1;
EPA 402-R-97-016, Rev. 1; and DOE/EH-0624, Rev. 1). The SAB Staff Office
announced this advisory activity and requested nominations for technical
experts to augment the SAB's Radiation Advisory Committee (RAC) in the
Federal Register (72 FR 11356; March 13, 2007).

MARSAME was developed collaboratively by the Multi-Agency Work Group (60
FR 12555; March 7, 1995) and provides technical information on
approaches for planning, conducting, evaluating, and documenting
radiological surveys to determine proper disposition of materials and
equipment (M&E). The techniques, methodologies, and principles that form
the basis of this manual were developed to be consistent with current
Federal limits, guidelines, and procedures.

The Panel met in an initial public teleconference meeting on Tuesday,
October 9, 2007. The meeting was intended to introduce the subject and
discuss the charge to the Panel, determine if the review and background
materials provided were adequate to respond to the charge questions
directed to the Panel, and agree on charge assignments for the
Panelists. A public meeting was scheduled on Monday, October 29 through
Wednesday, October 31, 2007, to receive presentations by the
multi-agency work group, consider the charge questions, and draft a
report in response to the charge questions pertaining to the draft
MARSAME manual. The Panel discussed the first public draft report
(December 17, 2007), in a December 21, 2007 public conference call. The
Panel discussed it's second public draft report (February 27, 2008) in
the March 10, 2008 public conference call. The April 24, 2008 draft
report was provided for a public quality review teleconference meeting
of the chartered Board on May 29, 2008. This report incorporates
suggestions made by the chartered SAB.

**2.3 EPA Charge to the Panel**

The EPA's Science Advisory Board (SAB) previously conducted the
scientific peer reviews of the companion multi-agency documents MARSSIM
(U.S. EPA/SAB, 1997; EPA-SAB-RAC-97-008, dated September 30, 1997) and
MARLAP (U.S. EPA/SAB, 2003b.; EPA-SAB-RAC-03-009, dated June 10, 2003).
The Federal agencies participating in those peer reviews considered the
process used by the SAB to be beneficial in assuring the accuracy and
usability of the final manuals. Subsequently, two consultations took
place for MARSAME (U.S. EPA/SAB, 2003a; EPA-SAB-RAC-CON-03-002, dated
February 27, 2003, and U.S. EPA/SAB, 2004; EPA-SAB-RAC-CON-04-001, dated
February 9, 2004). These are now being followed by a request from EPA
ORIA on behalf of the four participating Federal agencies that the SAB
conduct this formal technical peer review of the draft MARSAME manual.

The following charge questions were posed to the SAB MARSAME Review
Panel (U.S. EPA, 2007b):

*1) The objective of the draft MARSAME is to provide an approach for
planning, conducting, evaluating, and documenting environmental
radiological surveys to determine the appropriate disposition for
materials and equipment with a reasonable potential to contain
radionuclide concentration(s) or radioactivity above background. Please
comment on the technical acceptability of this approach and discuss how
well the document accomplishes this objective. In particular, please*

> *a) Discuss the adequacy of the initial assessment process as provided
> in MARSAME Chapter 2, including the new concept of sentinel
> measurement (a biased measurement performed at a key location to
> provide information specific to the objectives of the Initial
> Assessment).*
>
> *b) Discuss the clarity of the guidance on developing decision rules,
> as provided in MARSAME Chapter 3.*
>
> *c) Discuss the adequacy of the survey design process, especially the
> clarity of new guidance on using Scenario B, and the acceptability
> of new scan-only and in-situ survey designs, as detailed in MARSAME
> Chapter 4.*
>
> *d) Discuss the usefulness of the case studies in illustrating new
> concepts and guidance, as provided in MARSAME Chapter 7.*

*2) The draft MARSAME, as a supplement to MARSSIM, adapts and adds to
the statistical approaches of both MARSSIM and MARLAP for application to
radiological surveys of materials and equipment.  Please comment on the
technical acceptability of the statistical methodology considered in
MARSAME and note whether there are terminology or application
assumptions that may cause confusion among the three documents. In
particular, please*

> *a) Discuss the adequacy of the procedures outlined for determining
> measurement uncertainty, detectability, and quantifiability, as
> described in MARSAME Chapter 5.*
>
> *b) Discuss the adequacy of the data assessment process, especially
> new assessment procedures associated with scan-only and in-situ survey
> designs, and the clarity of the information provided in Figures 6.3
> and 6.4, as detailed in MARSAME Chapter 6.*
>
> *c) Discuss the usefulness of the case studies in illustrating the
> calculation of measurement uncertainty, detectability, and
> quantifiability, as provided in MARSAME Chapter 7.*

*3) The draft MARSAME includes a preliminary section entitled Roadmap as
well as seven appendices. The goal of the Roadmap is to assist the
MARSAME user in assimilating the information in MARSAME and determining
where important decisions need to be made on a project-specific basis.
MARSAME also contains appendices providing additional information on the
specific topics. Does the SAB have recommendations regarding the
usefulness of these materials?*

# RESPONSE TO THE STATISTICS ELEMENTS OF THE CHARGE QUESTIONS

Detailed discussions of statistical analysis related to experimental
design and hypothesis testing permeate the otherwise non-mathematical
guidance for M&E surveys. The Panel response and comments specifically
addressed to statistical analysis are compiled in Appendix A rather than
scattering them throughout this review. Appendix A consists of an
introduction that describes the view of the Panel, followed by specific
reviewer responses based on these reviews. All related responses to
individual charge questions, notably for charge questions 1b, 1c, and
2a, are referred to Appendix A.

The Panel recommends that topics presented in Appendix A be included as
a separate chapter that appears early in the MARSAME manual. This will
serve to consolidate many of the important statistical concepts that now
are now scattered throughout several chapters. The first-time user will
then become familiar with statistical considerations that are the
backbone for the MARSAME process.

MARSAME contains many suggested equations for designing and interpreting
survey procedures (e.g., Tables 5.1, 5.2). The equations are derived
from sound statistical principles, but can lead to incorrect conclusions
if the underlying assumptions in the derivations are not satisfied. Not
every equation needs to be derived in detail, but the assumptions and
sampling requirements to implement specific equations should be
thoroughly identified and explained.

Classical hypothesis testing procedures require specification of a null
hypothesis and values of α and β that quantify boundaries for type I and
type II errors. The selection of these values provides a measure of
tolerance for uncertainty and assurance that the ultimate goals relating
to risk are satisfied. The existing discussion in Chapter 4 is too vague
to provide guidance on how these values should be selected and should
either be more specific or \-- if this is considered to be beyond the
scope of MARSAME -- refer to sources of detailed guidance on the
selection of α and β.

In the design of disposition surveys, the manual discusses determination
of measurement uncertainty, detectability, and quantifiability in terms
of MQO requirements. These MQO values should be organized and presented
for individual types, such as *in situ*, scan only, or MARSSIM survey
types.

# 4. RESPONSE TO CHARGE QUESTION 1: PROVIDING AN APPROACH FOR PLANNING, CONDUCTING, EVALUATING AND DOCUMENTING ENVIRONMENTAL RADIOLOGICAL SURVEYS TO DETERMINE THE APPROPRIATE DISPOSITION FOR MATERIALS AND EQUIPMENT 

**4.1 [Charge Question 1:]{.underline} *The objective of the draft
MARSAME is to provide an approach for planning, conducting, evaluating,
and documenting environmental radiological surveys to determine the
appropriate disposition for materials and equipment with a reasonable
potential to contain radionuclide concentration(s) or radioactivity
above background. Please comment on the technical acceptability of this
approach and discuss how well the document accomplishes this
objective.***

The MARSAME manual impresses the Panel as an excellent technical
document for guiding an M&E survey. Regarding CQ 1, the Panel recommends
greater detail in describing the "alternate approaches or modification"
for applying MARSAME, as discussed in Chapter 1, lines 50 -- 56. For
example, the option of decontaminating the M&E as part of the process
when considering alternate actions appears to be missing. The Panel also
recommends making the manual more accessible to interested
non-specialists, notably project managers and other decision makers.
Such non-specialists generally are not included in the intended
"technical audience having knowledge of health physics and an
understanding of statistics," with further capabilities also described
in Chapter 1, lines 187 -- 194. The following itemized recommendations
elaborate on these points.

**RECOMMENDATION 1-1:** Create a sub-section for the discussion that
begins in Chapter l, line 49, to present clearly the concept of simple
alternatives to what may appear to the reader to be a major undertaking.
Also, in lines 103-111 further define 'release' *vs*. 'interdiction' to
clarify the distinction between the terms. Follow these paragraphs with
sufficient detail and references to later chapters to assure the reader
that when M&E is reasonably expected to have little or no radioactive
contamination, it can be processed without excessive effort under the
MARSAME system. One approach identified subsequently is applying
standard operating procedures (SOP's). Categorization as non-impacted or
as class 3 M&E based on historical data also can lead to an
appropriately simple process.

**RECOMMENDATION 1-2:** Insert a sub-section in Chapter 1 and in
appropriate subsequent chapters to consider various degrees of M&E
decontamination as part of the available options associated with a
MARSAME survey. Storage for radioactive decay can be an option for
decontamination.

**RECOMMENDATION 1-3:** Insert a paragraph after Chapter 1, line 196, to
address use by persons less skilled professionally than defined in a
preceding paragraph. Reference to Appendices B, C, and D, would be
helpful for such persons. Adding an appendix that includes portions of
the MARSSIM Roadmap and Chapters 1 and 2 could provide suitable
background information without requiring that all of MARSSIM be read.
Presentation of training courses for managers and other generalists with
responsibility for MARSAME radiation surveys would be most helpful.

**4.2 [Charge Question 1a:]{.underline} *Discuss the adequacy of the
initial assessment process as provided in MARSAME Chapter 2, including
the new concept of sentinel measurement (a biased measurement performed
at a key location to provide information specific to the objectives of
the Initial Assessment).***

The initial assessment (IA) process is useful as described. That many
measurements made throughout the MARSAME process could be biased should
be obvious to the radiation protection and survey professional.
Additional information sources cited below could be helpful.

Sentinel measurements, as described for the IA process of MARSAME have
been widely applied, although not necessarily designated by that name.
They are rational and useful for obtaining an IA of the type and
magnitude of radioactive contaminants although they may not have been
randomly selected and, hence, are biased by definition. These
measurements and their applicability and limitations are well described
in the document, and their use is clear. In fact, wider application
appears practical.

**RECOMMENDATION 1a-1:** Add to the information sources in Chapter 2,
lines 104 -- 115, the files (inspection reports, incident analyses, and
compliance history) maintained by currently and formerly involved
regulatory agencies. Discussion with agency staffs, especially their
inspectors, also could be fruitful.

**RECOMMENDATION 1a-2:** The listing of complexity attributes in Table
2.1 could include Toxic Substances Control Act (TSCA) materials and
hazardous waste.

**RECOMMENDATION 1a-3:** In Chapter 1, lines 253 -- 259, MARSAME should
recognize that sentinel measurements are important because they may
represent the entire historical record available for IA. Moreover, the
measurements may have been so well planned that considering them
"limited data" is misleading without a clear definition of terms.
Sentinel measurements are particularly useful to evaluate assumptions
based on process knowledge. In Chapter 2, lines 277 -- 280, design of a
preliminary survey for radioactive contaminants to fill knowledge gaps
often depends on the availability of data from sentinel measurements. In
some instances, the physical shape of the M&E may limit further survey
to sentinel measurements. On the other hand, the MARSAME Manual draft,
line 258, is correct in stating that sentinel measurements should not be
used alone to justify categorization of M&E as non-impacted, especially
when geometric or non-homogeneity limitations in radiation detection are
suspected.

**4.3 [Charge Question 1b:]{.underline} *Discuss the clarity of the
guidance on developing decision rules, as provided in MARSAME Chapter
3.***

This chapter, devoted to developing decision rules, is very useful. The
decision rules are clear. The Panel has the following recommendations
concerning (1) distinction among surface removable, surface fixed, and
volumetric radioactive contamination; (2) presentation of regulations
and guidance that address these contaminant forms; and (3) the
mathematically complex aspects of measurement method uncertainty,
detection capability, and quantification capability. With regard to the
latter, Chapter 3, lines 567 -- 622 takes the MARSAME presentation from
broad guidance to specific statistical tutorial, which raises
difficulties for some general readers and questions for some
professionals.

**RECOMMENDATION 1b-1:** The regulations or guidance for radionuclide
clearance that define the action levels (AL) discussed in Chapter 3,
lines 118 -- 120, and listed in Appendix E should be sufficiently
inclusive to apply to the usual M&E handled by users with regard to both
non-fixed ( removable) surface contamination and volumetric (distributed
throughout the material) contamination. Tabulate or cite all other known
pertinent regulations and guides for this purpose. To the non-fixed
surface contamination regulations included in Table E.2 by DOE and Table
E.3 by NRC, add the Department of Transportation regulation (U.S. DOT,
49CFR173.443), and guides by states such as New Jersey (State of New
Jersey, 2007) and Nevada (State of Nevada, 2001). Include guidance for
volumetric contamination clearance, summarized in Table 5.1 of NCRP
(2002) from reports of national and international standard-setting
groups.

**RECOMMENDATION 1b-2:** Information that guides decisions for
radioactively contaminated M&E, listed in Chapter 3, lines 141 -- 147,
should include measurements of removable *vs.* fixed surface
contamination to match the distinctions specified in Tables E.2 and E.3.
Insert sub-sections that discuss the implications of planning for and
responding to measurement of removable *vs.* fixed and surface *vs.*
volumetric radioactive contamination and the subsequent disposition of
M&E according to this categorization (see also RECOMMENDATIONS 2b-3 and
1d-3 for discussion of removable radioactive contaminants).

**RECOMMENDATION 1b-3:** Maintain the more general tone of MARSAME
throughout Chapter 3 while moving detailed discussions of statistical
aspects to a separate chapter (see also

RECOMMENDATIONS 1c-1 and 2a-1). This approach could remove concerns such
as why the Minimum Detectable Concentration (MDC) is recommended for the
Measurement Quality Objective (MQO) in Chapter 3, lines 593 -- 597,
instead of the Minimum Quantifiable Concentration (MQC), and how item #1
differs from item #3 on lines 609 -- 617.

**4.4 [Charge Question 1c:]{.underline} *Discuss the adequacy of the
survey design process, especially the clarity of new guidance on using
Scenario B. and the acceptability of new scan-only and in-situ survey
designs, as detailed in MARSAME Chapter 4.***

With the exception of Section 4.2, Statistical Decision Making, Chapter
4 is easily understood by the general reader. Classification of M&E is
an effective and helpful process. The Disposition Survey Design and
Documentation sections are well prepared. Further discussion would help
in addressing problems associated with complex geometric or
non-homogeneous distributions of the radioactive contamination relative
to the detector. These are of particular interest when using scanning or
*in situ* detection methods, and could be demonstrated effectively in
the illustrative example concerning rubble disposal of Section 7.3.

Regarding statistical decision making, the concepts of hypothesis
testing and uncertainty *per se* are readily understood. However, the
aspects of uncertainty with default significance levels and the
resulting gray area and discrimination limits (DL) leading to minimum
quantifiable concentrations (MQC) are not so readily assimilated.
Extensive consideration of the statistical approach is attached to this
review as Appendix A.

**RECOMMENDATION 1c-1:** In the organization of MARSAME, instead of the
current mixture of general guidance about surveillance with detailed
presentations of statistical matters, retain in each chapter only a
brief and less detailed discussion of statistics. Collect the
mathematical discussion in a separate chapter, as proposed above.
Chapter 19, Measurement Statistics, in MARLAP should serve as example.
The separation will serve both the specialist in statistics, who will
appreciate the exposition in the newly added chapter, and readers with
less training in statistics who can follow the general import of the
MARSAME approach in the existing chapters.

**RECOMMENDATION 1c-2**: The MARSAME manual has emphasized disposition
options that, after identification and segregation, lead directly to the
disposition survey. Conditioning of the M&E, such as vacuuming, wiping
down, chemical etching, and other forms of decontamination should be
encouraged for meeting disposition options (see also RECOMMENDATION
1-2). Preliminary measurements are useful for this purpose. The MARSAME
manual should provide more detail on these approaches and encourage them
as an As Low As Reasonably Achievable (ALARA) policy.

**4.5 [Charge Question 1d:]{.underline} *Discuss the usefulness of the
case studies in illustrating new concepts and guidance, as provided in
MARSAME Chapter 7.***

Case studies can be immensely beneficial for clarifying the MARSAME
process and guiding the user, but members of the multi-agency work group
informed the Panel that Chapter 7 does not contain case studies but
rather invented illustrative examples. The latter usually are not as
instructive as case studies because they lack the element of reality,
but can be helpful if created carefully to represent actual situations.

**RECOMMENDATION 1d-1:** Delete or replace the example for Standard
Operating Procedure (SOP) use in Section 7.2. Given the good discussion
in Section 3.10 for improving an SOP within the MARSAME framework, the
example of applying SOP's at a nuclear power station appears to
contribute little.

**RECOMMENDATION 1d-2:** The example in Section 7.3 of mineral
processing of concrete rubble is instructive, but the reader should be
informed that many more measurement results than those listed in Table
7.3 are obtained under actual conditions and must be evaluated before
making decisions. The radionuclide concentrations reported in Chapter 7,
lines 213 -- 214, should be confirmed as typical values or replaced by
such values, because readers may apply them as default values. For the
same reason, the AL taken from a U.S. Nuclear Regulatory Commission
document (NUREG-1640;U.S. NRC, 2003) should be identified as a specific
selection, not a general limit. Inserting boxes with interpretive
comments would help the reader to understand the process used for
illustration and the logic leading to the decisions.

**RECOMMENDATION 1d-3:** Insert an introductory statement to place in
context the length of the 21-page example devoted in Section 7.4 to a
simple baseline survey of a rented front loader, to avoid discouraging
the reader from applying it. This statement should explain that these
details are needed to describe the survey process, but that the actual
work is brief. This survey provides an opportunity to present the
benefit of sentinel measurements and the comparison of removable with
fixed surface contamination. An actual case history undoubtedly would
show these and also contain a table of survey measurements.

**RECOMMENDATION 1d-4:** Include in each of the illustrative example
headings a statement that they are demonstrating the MARSAME process.

# 5. RESPONSE TO CHARGE QUESTION 2: COMMENTS ON THE STATISTICAL METHODOLOGY CONSIDERED IN MARSAME 

[]{#__RefHeading___Toc143507193 .anchor}**5.1 [Charge Question \#
2:]{.underline}** ***The draft MARSAME, as a supplement to MARSSIM,
adapts and adds to the statistical approaches of both MARSSIM and MARLAP
for application to radiological surveys of materials and equipment.
Please comment on the technical acceptability of the statistical
methodology considered in MARSAME and note whether there are terminology
or application assumptions that may cause confusion among the three
documents.***

MARSAME contains tables and text that carefully compare the three
documents and identify consistencies and differences. To Panel members
familiar with the three documents, application of the statistical
methodology in MARSAME appears to match that used in MARSSIM and MARLAP.

The statistical methodology applied in MARSAME is acceptable and not
confusing when all three documents are read. Application of comments in
Appendix A to this report and consolidation of the mathematical aspects
of MARSAME in a single chapter as recommended below should enhance use
of MARSAME.

A shift appears to have occurred from use of the Data Quality Objective
(DQO) terminology of MARSSIM to the Measurement Quality Objective (MQO)
of MARSAME, but the principle is understandable. Clearly, MARSAME has
close connections to MARSSIM in surveys of M&E at MARSSIM sites. The
manual also addresses M&E that is to be moved onto or from a site for
various reasons, including - - but not necessarily - - processing and
surveying the site subject to MARSSIM.

**5.2 [Charge Question \# 2a:]{.underline}** *D**iscuss the adequacy of
the procedures outlined for determining measurement uncertainty,
detectability, and quantifiability, as described in MARSAME, Chapter
5.***

The presentation for determining uncertainty, detectability, and
quantifiability in Chapter 5, as well as aspects of this discussion in
Chapters 4 and 6, follows the well-developed path in MARSSIM and MARLAP
and is essential to the disposition survey planner. The Panel believes
that the outlined procedures are adequate, but that correct application
by the user requires (1) previous reading of MARSSIM and MARLAP, and (2)
the expertise and knowledge specified in Chapter 1, lines 189 -- 194.

**RECOMMENDATION 2a-1:** Enable the reader to understand the topics in
Chapter 5 more clearly by separating the entire mathematically detailed
statistical exposition in a chapter that could be entitled "Review of
Experimental Design and Hypothesis Testing." Appendix G can be included
in this chapter. The chapter can be placed before Chapter 4. All
sections currently in Chapters 4 -- 6 that discuss generalized aspects
of these topics, including measurement uncertainty, detectability, and
quantifiability, can be kept in place; reference should be made to the
technical discussions, equations, and tables in the new chapter.

**RECOMMENDATION 2a-2:** Consider the comments made in Appendix A
concerning the topics of experimental design, hypothesis testing, and
the statistical aspects of uncertainty in preparing the separate chapter
suggested above.

**RECOMMENDATION 2a-3:** Move the discussion on setting MQOs, in
Sections 5.5 thru 5.9, to Chapter 4 on Survey Design. Organize a summary
or guide that focuses on the procedures for setting MQOs and for
determining uncertainty, MDC, and MQC. The ability to set Measurement
Quality Objectives (MQOs) is an important element of the MARSAME
process, but the discussion involving the implementation of MQOs in the
design of the three survey types may confuse the reader. Aspects of
implementation are immersed in details defining, explaining, and
deriving theoretical concepts

**5.3 [Charge Question \# 2b:]{.underline}** ***Discuss the adequacy of
the data assessment process, especially new assessment procedures
associated with scan-only and in-situ survey designs, and the clarity of
the information provided in Figures 6.3 and 6.4.***

The data assessment process is appropriate, carefully presented, and
thoroughly explored. The advice is pertinent and the examples are
helpful.

The Panel discusses statistical considerations in Appendix A. The
information presented in Figures 6.3 and 6.4 is clear (See Figures 1 and
2, below), but minor changes, shown in the following two revised Figures
are proposed.

The Panel noted above the importance of distinguishing in all MARSAME
chapters among contamination that is (1) removable from the surface, (2)
fixed to the surface, or (3) volumetric. Smear surveys (wipe tests) are
an integral part of an M&E survey because of the potential radiation
dose from removable radionuclides that can spread from M&E surfaces and
be inhaled and ingested. Removable surface contamination is included in
DOE regulations in Table E.2 and NRC regulations in Table E.3, as well
as DOT regulations and International Atomic Energy Agency (IAEA)
guidance. Multi-agency working group members expressed reluctance about
including in MARSAME a survey technique that they consider to be poorly
reproducible for defining the removable radionuclide amount per area.
The Panel response is that insufficiently discussing wipe tests is
unrealistic and misleading. Each type of measurement has its own
uncertainty. A reasonable approach is to begin with the instruction in
U.S. DOT 49CFR173.443 "wiping an area of 300 cm^2^ ... with an absorbent
material ... using moderate pressure" that "sufficient measurements
shall be taken in the most appropriate locations to yield a
representative assessment" and then provide guidance on defining and
controlling variability.

**RECOMMENDATION 2b-1:** In Fig. 6.3 (See Figure 1 below, which reworks
Fig. 6.3), clarify the distinction of a MARSSIM-type survey by moving
"Start" to immediately above the decision point "Is the Survey Design
Scan-only or *In situ*?" and then connecting this to an inserted
decision diamond "Is the AL equal to zero or background?". A "yes" leads
to "Requires scenario B ..." and a "no" leads to "Disposition Decision
Based on Mean ...."

**RECOMMENDATION 2b-2:** In Fig. 6.4 (See Figure 2 below, which reworks
Fig. 6.4), for a more consistent presentation, insert a decision diamond
after both "Perform the Sign Test" and "Perform the WRS Test" that says
"Scenario A," followed by a "yes" or "no" leading to the two "Scenario
A" and "Scenario B" branches at both locations.

![](media/image3.jpeg){width="6.054166666666666in"
height="6.367361111111111in"}

**Figure 1 -- Suggested Changes for Figure 6.3 from MARSAME Manual for
Interpretation of Survey Results for Scan-Only and In-Situ Surveys**

![](media/image4.jpeg){width="4.715972222222222in"
height="6.054861111111111in"}

**Figure 2 -- Suggested Changes for Figure 6.4 from MARSAME Manual for
Interpretation of Results for MARSSIM-Type Surveys**

**RECOMMENDATION 2b-3:** To counteract the discomfort of Multi-agency
working group members with the qualitative aspect of wipe tests, the
MARSAME manual could recommend

evaluations of the removable radionuclide fraction measured by wipe test
for the surveyed M&E.

These evaluations can include, for example, sequential smears at a given
location at the M&E, or

smears at adjoining locations performed with different material and
pressure, by different

persons, and for different radionuclides. Refer to State of Nevada
(2001) and State of New Jersey (2007) for a description of the process,
to Rademacher and Hubbell (2008) pp. 10, 16 for an application to
radiological monitoring, and to U.S. EPA (2007a) for more general
applications of the wipe test.

**RECOMMENDATION 2b-4:** Insert sub-sections in all chapters to address
implementation and assessment of survey processes to distinguish between
surface and volumetric contamination (i.e., measurement after surface
cleaning or observing the effect of counting geometry) and between
removable and fixed surface contamination (i.e., wipe test results
compared to total surface activity). These types of contamination are
described in Chapter 1, lines 127 -- 152, but their implications should
be considered throughout the MARSAME manual. Concerns in measuring
volumetric contamination include characterizing non-uniformly
distributed radionuclides and quantifying radionuclides that emit no
gamma rays.

**5.4 [Charge Question \# 2c:]{.underline}** ***Discuss the usefulness
of the case studies in illustrating the calculation of measurement
uncertainty, detectability, and quantifiability as provided in MARSAME
chapter 7.***

As stated in the response to CQ 1d, case studies are invaluable in
guiding the user through complex operations. A useful case study
discussion with references is available in ITRC (2008). The illustrative
examples given instead of case studies in MARSAME lack the realistic
data accumulation that permits estimation of uncertainty. Excessively
detailed derivations of equations for calculation are shown in Chapter
7, lines 579 -- 628, 658 -- 665, 682 -- 689, and 1133 -- 1150. For
discussions related to uncertainty, refer to Appendix A.

**RECOMMENDATION 2c-1:** Move the detailed derivations, including
partial derivatives, identified above to the newly added separate
chapter recommended for discussion of experimental design and hypothesis
testing.

**RECOMMENDATION 2c-2:** Use illustrative examples to demonstrate any
MARSAME guidance that the multi-agency work group considers difficult to
follow. These may include approximating uncertainty (see Chapter 5),
distinctions such as interdiction *vs.* release, and applying scenarios
A *vs.* B.

**RECOMMENDATION 2c-3:** Use Sections 7.4 and 7.5 to illustrate the
benefit of wipe tests for determining removable radioactive surface
contaminants. Experience suggests that the contaminant usually is in
this form on M&E such as earth-moving equipment.

# 6. RESPONSE TO CHARGE QUESTION 3: RECOMMENDATIONS PERTAINING TO THE MARSAME ROADMAP AND APPENDICES 

**[Charge Question 3:]{.underline} *The draft MARSAME includes a
preliminary section entitled Roadmap as well as seven appendices. The
goal of the Roadmap is to assist the MARSAME user in assimilating the
information in MARSAME and determining where important decisions need to
be made on a project-specific basis. MARSAME also contains appendices
providing additional information on the specific topics. Does the SAB
have recommendations regarding the usefulness of these materials?***

The Roadmap is crucial in guiding the reader through a document as
complex as MARSAME. The appendices are useful in various ways, such as
providing information compilations and statistical tables, and avoiding
the need to seek this information in MARSSIM and MARLAP. Also necessary
to the reader are the acronyms and abbreviations; symbols, nomenclature,
and notations; and glossary. The following Recommendations are intended
to enhance their use.

**RECOMMENDATION 3-1**: Roadmap Figure 1 connects the MARSAME chapters
in terms of the Data Life Cycle. Consider establishing an analogous
connection with Roadmap Figures 2, 3, 5, 6, 7, and 8. At present, the
only Roadmap figures connected to each other are Fig. 2, 3, and 4, and 7
with 8.

**RECOMMENDATION 3-2:** Consider assisting project managers by
highlighting major operational decision points in the roadmaps.

**RECOMMENDATION 3-3:** The roadmap should ensure that the primary
components of the process are identified, their relationship to one
another is depicted, and the boundaries of application are well-defined,
in accord with the DQO process. Figure 3 provided below could be used in
the MARSAME roadmap to illustrate application of the DQO process in the
MARSAME manual. Realize also that the DQO process is iterative, so that,
as in the case of MARSSIM, the MARSAME program should have the potential
to improve and update the manual.

**RECOMMENDATION 3-4:** Indicate in the body of the text that Appendices
B, C, and D are useful overviews of the environmental radiation
background, sources of radionuclides, and radiation detection
instruments, respectively, for managers and generalists; they may be too
general for the experienced health physicist to whom the manual is
addressed.

**RECOMMENDATION 3-5:** Insert a table with action level (AL) guidance
for volumetric radionuclide contamination in Appendix E (see
RECOMMENDATION 1b-1).

**RECOMMENDATION 3-6:** Either move Appendix G into the new chapter on
experimental design and hypothesis testing or indicate its relation to
that new chapter.

**RECOMMENDATION 3-7:** Move the Glossary to the front to join the
tables of acronyms and of symbols.

**RECOMMENDATION 3-8:** Expand the definition of 'Interdiction' in the
glossary to clarify its application to receiving or disposing of M&E.

.

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 

# 7. RECOMMENDATIONS BEYOND THE CHARGE

**RECOMMENDATION C-1:** In Chapter 3, discuss in the recommended
separate chapter on statistics any decisions leading to selecting the
degree of confidence, embedded in the choice of significance level α and
β values. Selection may be a matter of the acceptable uncertainty
specified by the agency that sets the action level.

**RECOMMENDATION C-2:** In Chapter 2, discuss the impact of survey cost
and needed skills, instruments, and time on the MARSAME effort. Brief
projects obviously need different designs than lengthy ones. Discuss
requirement and program for data retention, especially in long projects
and when contractors are replaced.

**RECOMMENDATION C-3:** In Chapter 6, discuss the options to be
considered and pursued when the plan proposed initially for M&E transfer
is rejected because of the observed contaminant levels.

**RECOMMENDATION C-4:** Provide an additional Appendix that summarizes
topics in MARSSIM and MARLAP that are important to the MARSAME manual
but are insufficiently described in it, or at least give page references
to the earlier documents. Such topics may include aspects of quality
assurance (e.g., validation and verification of results), data
reliability affected by sample dimensions, measurement frequency, and
detector characteristics. Consider also the effect of non-random
variability in measurement (e.g., fluctuating geometry or monitor
movement rate).

REFERENCES

[Federal Register Notice Citations:]{.underline}

**FR**, Vol. 60, March 7, 1995, p. 12555

**FR**, Vol. 72, No. 48, March 13, 2007, p.11356

**FR**, Vol. 72, No. 184, September 24, 2007, pp. 54255 -- 54257.

**FR,** Vol. 72, NO.89, pp. 25695, May 7, 2008

ITRC, 2008 ***"Decontamination and Decommissioning of Radiologically
Contaminated Facilities,"*** Interstate Technology & Regulatory Council,
Radionuclides Team, Washington, DC 20001
(<http://www.itrcweb.org/Documents/RAD5.pdf>)

NCRP, 2002 ***"Managing Potentially Radioactive Scrap Metal,"*** Report
#141, National Council on Radiation Protection and Measurements,
Bethesda, MD 20814.

Rademacher, Steven E. and Joshua L. Hubbell, 2008 ***"Boeing Michigan
Aeronautical Research Center (BOMARC) Missle Shelters and Bunkers
Scoping Survey Report,"*** Vols. 1 &2, United States Air Force Institute
for Operational Health, Surveillance Directorate, Radiation Surveillance
Division, Brooks City-Base, TX, IOH-SD-BR-SR-2008-000\_, February 2008

State of Nevada, 2001 ***"Division of Environmental Protection Dry Wipe
Sampling,"*** Oct. 16, 2001 (7 pages)
<http://ndep.nv.gov/fallon/dry.pdf>

State of New Jersey, 2007 ***"Radiological Emergency Response Plan,"***
SOP 361 Wipe Sampling, Rev. 9, May 2007

U.S. DOT, 49CFR173.443 ***"Package and Vehicle Contamination Limits.
Minimum Required Packaging for Class 7 (Radioactive) Materials"***

U.S. EPA, 2000 and 2001 ***"Multi-Agency Radiation Survey and Site
Investigation Manual***" (MARSSIM). NUREG-1575, rev. 1; EPA
402-R-97-016, Rev. 1, DOE/EH -- 0624, Rev. 1, August 2000 and June 2001
update

U.S. EPA, 2006 ***"Guidance on Systematic Planning Using the Data
Quality Objectives Process, EPA QA/G4,"*** EPA/240/B-06/001, February
2006

U.S. EPA, 2006b ***"Multi-Agency Radiation Survey and Assessment of
Materials and Equipment Manual (MARSAME), Draft Report for Comment,"***
NUREG-1575, Supp. 1; EPA 402-R-06-002; DOE/EH-707, December 2006

U.S. EPA, 2007a ***"A Literature Review of Wipe Sampling Methods for
Chemical Warfare Agents and Toxic Industrial Chemicals,"***
EPA/600/R-07/004, January 2007
(<http://www.epa.gov/NHSRC/pubs/reportWipe042407.pdf> )

U.S. EPA, 2007b Memo from Elizabeth A. Cotsworth, Director, Office of
Radiation and Indoor Air (ORIA) to Vanessa Vu, Director, Science
Advisory Board Staff Office, and entitled ***"Review of the Draft
Multi-Agency Radiation Survey and Assessment of Materials and Equipment
Manual,"*** October 23, 2007

U.S. EPA/SAB, 1997 ***"An SAB Report: Review of the Multi-Agency
Radiation Survey and Site Investigation Manual (MARSSIM),"*** Prepared
by the Radiation Advisory Committee (RAC) of the Science Advisory Board,
EPA-SAB-RAC-97-008, Sept. 30, 1997

U.S. EPA, 2004 ***"Multi-Agency Radiological Laboratory Analytical
Protocols (MARLAP) Manual,"*** NUREG-1576; EPA 402-B-04-001A; NTIS
PB2004-10521, July 2004

U.S. EPA/SAB, 2003a ***"Consultation on Multi-Agency Radiation Site
Survey Investigation Manual (MARSSIM) Supplements for Materials and
Equipment (MARSAME): A Science Advisory Board Notification of a
Consultation,"*** EPA-SAB-RAC-CON-03-002, February 27, 2003

U.S. EPA/SAB, 2003b ***"Multi-Agency Radiological Laboratory Analytical
Protocols (MARLAP) Manual: An SAB Review of the MARLAP Manual and
Appendices by the MARLAP Review Panel of the Radiation Advisory
Committee (RAC) of the U.S. EPA Science Advisory Board (SAB),"***
EPA-SAB-RAC-03-009, June 10, 2003

U.S. EPA/SAB, 2004 ***"Second Consultation on Multi-Agency Radiation
Site Survey Investigation Manual (MARSSIM) Supplements for Materials &
Equipment (MARSAME): A Science Advisory Board Notification of a
Consultation,"*** EPA-SAB-RAC-CON-04-001, February 9, 2004

U.S. NRC, 1997 ***"A Nonparametric Statistical Methodology for the
Design and Analysis of Final Status Decommissioning Survey,"*** Draft
Report for Comment, Washington, DC, Nuclear Regulatory Commission (NRC)
NUREG-1505, August, 1995.

U.S. NRC, 2003 "***Radiological Assessment for Clearance of Materials
from Nuclear Facilities,"*** Nuclear Regulatory Commission (NRC), Office
of Nuclear Regulatory Research, Washington DC, NUREG-1640, Vols. 1-4,
June 2003.

**[Web-based Citations and Hotlinks]{.underline}**

MARSSIM: <http://epa.gov/radiation/marssim/index.html>

MARLAP: <http://epa.gov/radiation/marlap/index.html>

MARSAME: [http://www.marsame.org](http://www.marsame.org/)

DRY WIPE SAMPLING PROTOCOL FOR THE STATE OF NEVADA:

State of Nevada, 2001 Division of Environmental Protection Dry Wipe
Sampling, Oct. 16, 2001 (7 pages) <http://ndep.nv.gov/fallon/dry.pdf>

[]{#__RefHeading___Toc143507226 .anchor}**APPENDIX A -- STATISTICAL
ANALYSIS -- AN INTRODUCTION TO EXPERIMENTAL DESIGN AND HYPOTHESIS
TESTING WITH SPECIFIC COMMENTS ON STATISTICS**

**A-1 An Introduction to Experimental Design and Hypothesis Testing:**

The general problem of designing a survey of the sort described in the
MARSAME document involves the following issues:

(1) Understanding the error properties of the measurement instrument and
    how they can be manipulated (by changing counting times or
    performing repeated measurements of the same radionuclide quantity,
    for example). Generally the measurement error can be well
    characterized by its standard deviation ~M.~ This value may be a
    constant (all measurements have the same standard deviation) or it
    may vary with radiation level (as in the behavior of an idealized
    radiation counter);

(2) Understanding the distribution of radionuclides in the population of
    equipment or material that is to be measured. This distribution can
    often be well characterized by a standard deviation ~S­~ which we may
    call the sampling standard deviation;

(3) Deciding upon the number of samples, N, from the distribution of
    radionuclide concentration that will be used in the detection
    problem;

(4) Specifying the null and alternative hypotheses to be examined; the
    symbol Δ represents the quantity of excess radionuclides equal to
    the difference between the null and the alternative hypothesis
    values;

(5) Specifying values of α and β that quantify acceptable limits for
    type I and type II errors;

(6) Determining, with fixed Δ and α, the power 1 -- β to reject the null
    hypothesis in favor of the alternative.

From a statistical standpoint, designing an experiment means finding
values of the sample size N and the detectable difference that will
control type I error and power, given the instrument's measurement error
properties and the sampling radionuclide concentration distribution.

In MARSAME, the null and alternative hypotheses generally concern the
true difference in radionuclide levels between a potentially
contaminated material or piece of equipment and the appropriate
background reference. In Scenario A, the null hypothesis is that the M&E
is at least as radioactive (over background) as some number called AL
(the action level), and the alternative is that the true exess
radionuclide level is less than AL. In Scenario B the null hypothesis is
that the M&E is at the action level (which usually equals the background
in scenario B) and the alternative hypothesis is that the M&E is over
the AL. These scenarios are further discussed below. The MARSAME manual
should note the interplay between α and 1 -- β. For a fixed study
design, power can be defined only in terms of α since power is the
probability of rejecting the null hypothesis at a given α. (Note that
terminology used here follows the MARSAME Glossary and list of Symbols,
Nomenclature and Notations.)

When a single measurement is taken, the variance of that measurement
will equal ![](media/image5.wmf). In some cases the sampling
distribution and thus ~S~ may be irrelevant to a MARSAME survey; for
example, there may be no spatial variability (when there is only 1 level
of radiation relevant to a small item). An important issue is how the
error properties of the instrument behave when repeated measurements of
the same equipment item or same portion of material are taken. For some
measuring instruments, it may be reasonable to assume that the average
of N measurements of the same unit will have standard deviation equal to
![](media/image6.wmf). This will be the case in an idealized radiation
counter, since performing additional measurements on the same sampling
unit (item) is equivalent to increasing the count times for that unit.
In other cases, inherent biases in measurement instruments may result in
a measurement error shared by all measurements.

When sampling variability occurs (so that ~S~ is not zero), the variance
of the mean of a random sample of N measurements of will have variance
somewhere in the range ![](media/image7.wmf)to![](media/image8.wmf). The
first of these corresponds to measurement errors that are completely
unshared and the second corresponding to measurement errors that are
completely shared due to imperfect calibration (for example, in the
"measured efficiency" of a monitor discussed in several places in the
manual). Generally, as more measurements are taken, the contribution of
the sampling variance to the variance of the mean tends to disappear,
whereas some or all of the contribution of the measurement error may
remain. The special case when 100% of a potentially contaminated
material is measured may be regarded as the limit when N -\> . Again,
some or all of the measurement error variance may still remain.

For most situations in MARSAME, the null hypothesis concerns the
difference between background levels and the level of contamination of
the M&E. Table 5.1 (in the current document) gives some special formulae
used when counts in time follow a Poisson distribution (so that the
variability of the counts of both background and the item of interest
depends on counting time and radiation level). In general, the variance
of the difference between sampled radioactivity and the estimate of
background will require special investigation as a part of the survey
design.

For simplicity, it is useful to denote the standard deviation of
measurement minus background as , which refers to the standard deviation
of the estimate (often termed the standard error) obtained from the
entire measurement method (involving either single readings, multiple
readings, scans of some or all of the material, etc.). This can be a
relatively complicated function of the underlying measurement and
sampling variability (which must include the uncertainties in the
estimate of background) that may require careful study to quantify
properly.

Once is determined, the power, 1-, of a study will depend upon two other
parameters, (1) the type I error rate and (2) the size of the assumed
true difference . If the standard error of the estimate, , is the same
for all radiation levels being measured, then the ratio / determines
power for a given value of α (otherwise a more complicated expression is
used, as in Table 5.1 of MARSAME). For known , we may specify the
"detectable difference " by fixing both the type I error and the power
1- and solving for . In the MARSAME manual, this detectable difference
is called the width of the "gray region." (Differences less than this
are only detectable with power less than the required 1- and hence are
"gray.") If the action level, AL, is defined to be the upper bound of
the "gray region," then the lower bound (AL- detectable difference ) is
called the "discrimination limit" (DL). Note that implicitly the
detectable difference and the DL depend upon the power, type I error
rate, and the standard error of the estimated . *One of the confusing
aspects of the MARSAME manual is that the DL is introduced long before
the concept of power or type I error*.

The two scenarios (A and B) considered in the report both assume that
the null hypothesis is at the action level, but differ in the direction
of the alternative hypothesis and generally in the value of AL. Under
scenario A, the alternative hypothesis is that the radiation level is
less than the action level (which is the upper limit above background to
be allowed) whereas under scenario B the alternative hypothesis is that
the radiation level is greater than the action level (which is typically
set to background). *Under scenario A the M&E is only deemed to be safe
for release if the null hypothesis is rejected, whereas under scenario B
the M&E is safe for release if the null hypothesis is **not** rejected.*

If under scenario A, for example, the true value of the radionuclide
level (or level above background) is less than or equal to DL then the
survey will have power 1- to reject the null hypothesis that the true
value is equal to the AL with type I error . Under scenario B, if the
value of true contamination-background is *greater* than the detectable
difference , then the study will again have power 1- to reject this null
hypothesis at type I error rate . Assuming that the standard error of
the estimate , does not depend upon the radiation levels being measured,
the formula for the "detectable" , given , and power 1- is

Detectable difference![](media/image9.wmf) (1)

Where ![](media/image10.wmf)and ![](media/image11.wmf) are the
corresponding critical regions for the standard normal random variable.
A somewhat more complicated formulae for is needed when is not
independent of radiation level as in Table 5.1; however, formulae (1)
gives a useful (conservative) approximation to the detectable difference
if we choose to be at its maximum likely value for either the null or
alternative hypothesis.

In general, use of equation (1) for the detectable difference requires
that the estimate of contamination (measurement -- background) be
approximately normally distributed. For radiation counters with long
count times and large values of N (when there is sampling variability as
well as measurement variability), this assumption is usually quite
appropriate. Because the width of (for fixed power and type I error)
depends on , it is important that an instrument or measurement technique
(and sampling fraction for spatially distributed contamination) is
selected which is sensitive enough (provides small enough ) so that the
detectable meets requirements (for example so that the DL is not set to
be too small in Scenario A, or that the upper range of the gray region
is not set too high above background in Scenario B).

In some situations (non-normal distributions, short count times), the
detectable will be larger than described in equation (1) and more
specialized statistical analysis may be needed. Such techniques as
segregation according to likely level of contamination may improve the
accuracy of equation (1), as will longer count times.

Hypothesis testing (accepting or rejecting the null hypothesis) involves
comparing an estimate of contamination level to a "critical value"
(termed S~c~ in the manual) which allows us to decide whether the
observed estimate is consistent with the null value (at a certain type I
error level) after taking account of the variability (i.e., ) of the
measurement. For Scenario A, this value is equal to S~c~ = AL -- Z~1-~ ,
and for Scenario B it is S~c~ = AL + Z~1-~ . By definition, power is the
probability, as computed under the alternative hypothesis, of rejecting
the null hypothesis; that is, the probability that the observed estimate
is less than (for scenario A) or greater than (for scenario B) the
critical value S~c~.

If the normality of the estimate is in doubt, then other approaches to
hypothesis testing may be needed. For example, while for long count
times the Poisson distribution can be approximated as normal for the
purpose of hypothesis testing, for short count times, specialized
formulae (see Section 5.7.1) may be needed to give a better
approximation to the distribution of (measured-baseline) for an
idealized radiation counter.

**A-2 Specific Comments:**

Section 3.8.1 describes "Measurement Method Uncertainty" but in somewhat
more vague terms than above. The intent of this section could be better
understood in reference to the suggested introduction to experimental
design and hypothesis testing.

All of Chapter 4 would be more comprehensible if it consistently
referred back to the suggested introduction to experimental design and
hypothesis testing.

Section 4.4.1.2 gives a recommendation for how much of an impacted
material should be scanned: it is not clear to what the value now refers
(eq 4-1). This appears to be the measurement error standard deviation
~M~ rather than the total standard deviation of the measurement method
(measurement method uncertainty). Presumably, this is giving a
recommendation that will keep the total measurement method uncertainty
bounded for a given level of measurement error (~M~).

The statistical concepts described earlier MARSAME are illustrated for
the first time in Figures. 4.2 and 4.3. It is unfortunate that even
though the concepts shown in the figures all relate to net
radioactivity, they are termed "level," "value" or "limit." This could
cause misinterpretation by someone who is preparing to establish a
survey design. An expansion of these figures to include several
additional parameters with some supplemental text would be helpful.

![](media/image12.wmf){width="3.625in"
height="2.5055555555555555in"}Recommendations for scenario A and B are
presented in Figs. A-1 and A-2. These embellished Figures with some
additional text should also eliminate the need to repeat this
information in Chapter 5, as in Figs. 5.2, 5.3, 5.4.

![](media/image13.wmf){width="4.25in"
height="3.0131944444444443in"}**Figure A-1. Scenario A**

**Figure A-2. Scenario B**

As mentioned above, the Action Level (AL) for net excess radioactivity
is used in defining the null hypothesis. However, the decision on
accepting the null hypothesis is not based on the numerical value of net
radioactivity at the AL. Rather, each sample is compared with the
Critical Value shown in the Figures. This ensures that the probability
for rejecting the

null hypothesis, when it is true, will not exceed . The Discrimination
Limit (DL) is the net

radioactivity in the sample where the probability of accepting the null
hypothesis, when it is false, is (i.e., the power for rejecting the null
hypothesis is 1-). The Gray area is the region of net radioactivity in
the sample where the statistical power to reject the null hypothesis,
when it is false, is less than 1-.

Application of Measurement Quality Objectives (MQOs) discussed in
Section 5.5 is an operational aspect of the MARSAME process. MQOs are
part of the Data Quality Objectives process (DQOs) used as a platform
for both the MARSSIM and MARSAME process. Use of MQOs was not
incorporated into the MARSSIM process, so it maintains a unique role to
MARSAME. The application of MQOs is fairly new to Decommissioning
planning. It was employed in MARLAP in 2004 for laboratory-based
measurements and now has been extended to field measurements in MARSAME.
The Guide to the Expression of Uncertainty in Measurement (GUM), which
forms the basis for much of the conceptual and statistical framework of
MQOs, was published by the International Standards Organization (ISO)
and the National Institute of Standards and Technology (NIST) in 1995
and 1994, respectively. The topic of MQOs may be unfamiliar to many
users of the MARSAME. For this reason, it is important to provide a
sound basis for the operational and statistical aspects of its use.

Some SAB MARSAME Review Panel comments, in the text and in this
Appendix, specifically address the theoretical foundations of the
underlying statistical assumptions used in the mathematical
relationships and equations. Other panel comments address the
application of MQOs from an operational standpoint. The identification
of MQOs for certain types of measurement cases and survey designs may be
confusing to readers unfamiliar to MQO applications. Considerable detail
in the manual is provided on defining, explaining, and deriving the
relevant theoretical concepts. The writers of the MARSAME manual should
ensure that operational information on the implementation of MQOs is not
too deeply embedded within the theoretical discussion. More distinction
should be placed on information applicable to identifying performance
characteristics, setting MQOs, and selecting appropriate measurement
methods. Effective use of the manual relies on the reader to be able to
apply MQOs to their specific measurement problem.

A summary or guide that organizes the measurement uncertainty,
detectability, and quantifiability requirements for each of the three
types of MARSAME surveys, including In-Situ, Scan-only, and
MARSSIM-type, would be beneficial to the user. The guide would collect
information on the selection of MQOs, which may be scattered throughout
the chapter, into one coherent presentation for ready reference. The
guide would be useful for designing MARSAME disposition surveys,
training activities and for reference when regulators evaluate the
measurement requirements of disposition survey plans.

The presentation of statistical formulations and derivations can be
quite detailed and extensive and, if not properly balanced with the
operational aspects of the guidance, may detract from the clear
presentation of the guidance to the target audience. It is important to
recognize that the manual is written for those directing and
implementing the process, interpreting results, and making decisions.
The operational aspects of the guidance address this broad audience
directly; however, there is an audience concerned with the scientific
and technical soundness of the procedures and the rigor for which the
process is founded. An appropriate balance between the presentation of
the operational aspects and the statistical foundations of the guidance
is paramount.

The intent of Section 5.5 would be made clearer as dealing with the
factors that impact the measurement error uncertainty as described in
more general terms in the suggested review of experimental design and
hypothesis testing. Apparently, however, ~M~ (the standard deviation of
a single measurement not taking into account spatial distribution of
materials or the variability of the background) is being confused with
the overall (total measurement method uncertainty taking these factors
into account). It is / , not / ~M~, that determines the overall power of
the experiment. The document should clearly differentiate these two 's.

Section 5.5.1 lines 289-293 seems to be confusing ~M~ with ~s~. It is
~s~ that, generally speaking, can be decreased by improving scan
coverage (not ~M~ if this includes "shared" error terms such as the
"variance of measured efficiency"). The new terminology *u~MR~*
apparently refers either to an estimate of the measurement error
uncertainty ~M~ or to overall but this is not made clear in this section
(and the requirement that *u~MR~ ~s~/3* makes no sense if ~S­~ can be
reduced to 0 by improving scan coverage).

The comments on line 302-303 seem to require that *u~MR~* estimates the
overall . Example 2 is confusing because the requirement that *u~MR~* be
a factor of 10 times smaller than seems to assume that *u~MR~* is an
estimate of ~M~­ rather than the overall uncertainty (this would be a
very stringent requirement indeed). Here one needs to focus not just on
~M­~ but rather on the total variability including ~S~­. If ~s~ can be
reduced to zero by scanning all of a material why is such a stringent
requirement made on ~M~?

Line 360 introduces new and not clearly defined uncertainties (u~c~­ and
~MR~). Example 5 is unclear, and needs to be tied to some general design
or hypothesis testing principles -- it just comes out of thin air as it
stands.

Section 5.6 is a good description of addressing measurement uncertainty
~M~ in certain special cases. One thing that could be clarified is that
~M­~ now refers to the error in measurement - background rather than just
the error in the measurement itself. At other points in the manual, ~M~
seems to refer rather to the variance of just the measurement.

All determinations of excess radioactivity are based on the difference
between a sample with an unknown amount of radioactivity, and an
appropriate control that may contain radioactivity not related to the
source of contamination. MARSAME does not provide very much information
on how to characterize properly the "background" radiation contained in
controls or "reference samples."

Tables 5.1 and 5.2 list equations to determine critical values, S~c~. A
sample is considered to contain radioactivity in excess of the control
if the "net" result is greater than the S~c~. The value of S~c~ is based
on the probability that the net result of a sample with no excess
radioactivity that will exceed S~c~, is equal to (i.e., false positive).
This is, in effect, an example of Scenario B described in Chapter 4.
This is expanded in Table 5.2 to the minimum detectable value, S~D~. It
is the smallest value of net radioactivity, MDC, that will yield an
observed measurement greater than S~c~ with a statistical power of 1-.
That is, the probability that a sample containing exactly the MDC will
be less than S~c~ is (i.e., false negative).

The equations in Tables 5.1 and 5.2 are used throughout MARSAME as
examples for estimating critical values S~c~ and MDC. These equations
are based on the Poisson assumption for counting statistics and
distribution of the difference between two random numbers that are
Poison distributed. In effect, this implies that an independent
measurement of a control is paired with each measurement of a sample.
S~c~ is based on the distribution of two random numbers selected from
the same distribution of background.

Although the equations are correct, it is not common to measure a
control for every sample of unknown contamination. This process of
comparing paired samples is rare. Generally, an estimate of background
radioactivity is established, and subtracted from every sample to
estimate the "net" count.

Tables 5.1 and 5.2 are used throughout MARSAME without any reference to
any assumptions that were used to derive the equations. There could be
serious implications in decisions relating to the presence of
radioactivity using S~c~ and hypothesis testing using MDC as the DL. On
the other hand, for most cases these equations might be satisfactory. It
will be important for the MARSAME manual to clarify this, and to provide
more details on how to measure and characterize "background" in controls
that are used to determine "net" activity.

Some examples are shown below. For this case, equations 5.1.1 (Currie)
and 5.1.3 (Stapleton) were used to compute S~c~. A Monte Carlo model was
used to estimate S~c~ for paired samples from the true background
distribution (MC) and also for a constant background, equal to the true
mean, that was subtracted from a random sample of background (MCB). For
these cases, = = 0.05. Fig. A-3 is for the case where the sample time
t~s~ and the background time t~b~ are equal and yield a mean count of
200. The abscissa is normalized to the value of S~c~ obtained from the
Currie equation.

This illustrates that S~c~ obtained from 5.1.1 does indeed come from a
distribution of paired samples which is simulated in MC. However the
value for S~c~ obtained by subtracting a constant value equivalent to
the mean value of background, MCB, is actually about 30% lower than S~c~
from the equations.

Fig. A-4 is for the case where the sample time t~s~ is 5 and the
background time t~b~ is 50. For this case, the background is estimated
with greater precision because t~b~ is large. With a constant background
to estimate background, the value of S~c~ is similar to that obtained
from the equations in Table 5.1; however both MCB and the Currie
equation yield a value of S~c~ that is somewhat lower that that obtained
from paired samples (MC) by Monte Carlo simulation.

![](media/image14.wmf){width="4.789583333333334in"
height="2.709722222222222in"}

**Figure A-3. Comparison of S~c\ ~**

![](media/image15.wmf){width="4.789583333333334in"
height="2.709722222222222in"}

**Figure A-4. Comparison of S~c~ for longer background counting period**

![](media/image16.wmf){width="4.102777777777778in"
height="2.3131944444444446in"}

**Figure A-5. S~c~ for a briefer background counting period**

![](media/image17.wmf){width="4.127083333333333in" height="2.5in"}

**Figure A-6. 1-β as function of % excess count above background**

Fig. A-5 is for the case where t~s~ is twice the value of t~b~. Values
obtained for S~c~ using the Currie equation are close to the value from
the Monte Carlo simulation for paired samples, but the estimate of S~c~
using constant value of background is low by about 40%.

Fig. A-6 shows an example of the statistical power, 1-, as a function of
the increasing amounts of radioactivity above background. The curve
starting on the ordinate at a statistical power, 1-, of 0.05 represents
the simulation for paired samples and the curve starting at the origin
represents the simulation when a constant value of background is
subtracted from the sample to form the net value. Without excess
radioactivity, for the paired samples is 0.05 and = 0.01 when background
is a constant. The two curves are identical when the excess
radioactivity corresponds to S~c~ and therefore = 0.5. The vertical line
corresponds to the value of MDC obtained from equation 5.2.1. Note that
the MDC, (1-) = 0.95, obtained from the simulation with constant value
for background is smaller than when using the assumption of paired
samples.

MARLAP provides additional modifications to estimating S~c~ when the
Poisson approximation may not be satisfied. However, it is not clear
that the concerns relating to the process of measuring controls or
reference materials have been eliminated.

Many equations have been suggested for designing and interpreting survey
procedures in MARSAME. The equations are derived from sound statistical
principles. They can lead to incorrect conclusions if the underlying
assumptions in the derivations are not satisfied. The Panel does not
recommend that each equation be derived in detail, but suggests that the
assumptions and sampling requirements needed to properly implement
equations be documented in MARSAME.

Section 5.8, Determining Measurement Quantifiability is a complicated
way of saying that must be small enough (and hence / large enough) for
the measurement method to have good power to reject the null hypothesis
that the level of radioactivity is at the AL for a reasonable (width of
the gray region). It also must give a reasonably narrow confidence limit
for the estimated value, i.e. where the width of the confidence limit is
small compared to the value of the AL.

One complication that is explicitly dealt with in the definition of the
MQC is that the measurement method uncertainty, i.e. , generally will
depend upon the (unknown) true level of radioactivity itself -- for
example a perfect counter has Poisson variance equal to its mean. Thus
the MDC is just the value, y0, of the radioactivity level for which the
ratio, k=y0/, is large (the manual recommends k=10). If y0 is small
relative to the action limit (between 10-50 percent of the AL is
recommended), then it is clear that (1) the detectable will be small
with respect to the action limit (i.e. the DL will be close to the AL)
and (2) confidence limits around an estimated value of radioactivity
will be narrow relative to the value of the AL. Saying this clearly
improves the intelligibility of this section.

Section 5.8.1 would be more intelligible if it first noted that it is
giving a computation of the MDC, y0, for a fixed k by a formulae for
that takes account of several factors which are combined into this one .
These factors are the length of the reading time for the source, the
length of reading time for the background, the true value of the
background reading, and an estimate of the variance of a "shared"
measurement error term, i.e. the measured efficiency of the monitor.

Section 6.2.1 has some confusing aspects: as described earlier, the gray
region is defined in terms of the power and type I error of the test
with a measurement method of total standard deviation . Sentences like
"Clearly MDCs must be capable of detecting radionuclide concentrations
or levels of radioactivity at or below the upper bound of the gray
region" seem tautological if the gray region is defined in terms of
detection ability; specifically in terms of power, type 1 error, and .

Lines 215-224 of Section 6.2.3 confuse by the statements about how
individual measurement results can be utilized for scan-only
measurements. The statement that "if disposition decisions will be made
based on the mean of the logged data, an upper confidence level for the
mean is calculated and compared to the UBGR," must be interpreted
carefully. If one did a standard test such as Wilcoxan or t-test) one
would ignore any uncertainty component resulting from variability in the
measurement process (i.e. measurement error shared by all measurements
that constitute the scan). Only if ~M~ has no shared components (or if
they are very small) would it make sense to do a standard statistical
test of the observed data alone. Specifically, the sample standard
deviation would underestimate the true measurement standard deviation if
a shared uncertainty (such as errors in the estimate of counting
efficiency) is incorporated in ~M~.

The recommendation (line 60) that for MARSSIM type surveys the sample
standard deviation can be used to generate a power curve also implicitly
assumes that no shared measurement error components exist. But this
contradicts the conclusion of line 223-224 that "Measuring 100% of the
M&E accounts for spatial variability but there is still an uncertainty
component resulting from variability in the measurement process." In
fact, all the discussion of selecting and performing a statistical test,
and drawing conclusions in the rest of Section 6 seems to be implicitly
assuming that there are no shared errors from measurement to
measurement. Is this the intention? If this was the intention, clarify
the confusing discussion in Section 5.5.1, lines 289-293. For example,
even if all measurements are less than the action level, this might not
really be enough information to conclude that the M&E meet the
disposition criterion.

Suppose all measurements are only somewhat less than the action level
but it is also known that the counting efficiency was not well
estimated. Ignoring the uncertainty in the counting efficiency could
lead to the wrong conclusion in this case, if the uncertainty in the
counting efficiency is indeed "shared error" over all the measurements.
In many places in this document, errors in counting efficiency or other
apparently shared measurement errors are mentioned (as on line 223-224),
but this issue seems to be ignored in most of Section 6. If the manual
assumes that such shared errors are small enough to be ignored, this
should be stated explicitly. (See also footnote 4 on page 6-17).

One possible resolution is to assume that the measurement of background
has exactly the same "shared" uncertainties (counter efficiencies, etc.)
as does the measurement of the radioactivity level in the M&E. In this
case, the shared uncertainties will be subtracted when the background is
subtracted from the level measured in the M&E. If this is meant, then it
should be stated clearly (and this should be highlighted in the any
initial "review of experimental design and hypothesis testing" when
discussing the various components included in ).

[]{#refheading___toc143507226 .anchor}APPENDIX B --ACRONYMS AND
ABBREVIATIONS

A Scenario [A]{.underline} for hypothesis testing

AL [A]{.underline}ction [L]{.underline}imit (or [L]{.underline}evel)

ALARA As Low As Reasonably Achievable

Maximum acceptable probability for Type I error rate (alpha)

AM [A]{.underline}rithmetic [M]{.underline}ean

Maximum acceptable probability for Type II error rate (Beta)

B Scenario [B]{.underline} for hypothesis testing

1- Numerical value of the statistical power to reject the null
hypothesis when it is true

CFR [C]{.underline}ode of [F]{.underline}ederal
[R]{.underline}egulations

CON [Con]{.underline}sultation

CQ [C]{.underline}harge [Q]{.underline}uestion (CQ1, CQ 2, CQ3)

Difference (Alternative -- Null hypothesis), also the Detectable
Difference

DFO [D]{.underline}esignated [F]{.underline}ederal [O]{.underline}fficer

DL [D]{.underline}iscrimination [L]{.underline}imit (also
[D]{.underline}iscrimination [L]{.underline}evel)

DLC [D]{.underline}ata [L]{.underline}ife [C]{.underline}ycle

DoD [D]{.underline}epartment [o]{.underline}f [D]{.underline}efense
(U.S. DoD)

DOE [D]{.underline}epartment [o]{.underline}f [E]{.underline}nergy (U.S.
DOE)

DOT [D]{.underline}epartment [o]{.underline}f
[T]{.underline}ransportation (U.S. DOT)

DQO [D]{.underline}ata [Q]{.underline}uality [O]{.underline}bjective

EH [E]{.underline}nvironmental Safety and [H]{.underline}ealth (U.S.
DOE/EH)

EPA [E]{.underline}nvironmental [P]{.underline}rotection
[A]{.underline}gency (U.S. EPA)

FR [F]{.underline}ederal [R]{.underline}egister

GUM [G]{.underline}uide to the Expression of [U]{.underline}ncertainty
in [M]{.underline}easurement

H~o~ Null [H]{.underline}ypothesis

IA [I]{.underline}nitial [A]{.underline}ssessment

IAEA [I]{.underline}nternational [A]{.underline}tomic
[E]{.underline}nergy [A]{.underline}gency

ISO [I]{.underline}nternational [S]{.underline}tandards
[O]{.underline}rganization

ITRC Interstate Technology & Regulatory Council

k Coverage Factor for Uncertainty

LBGR [L]{.underline}ower [B]{.underline}ound of the [G]{.underline}ray
[R]{.underline}egion

MARLAP [M]{.underline}ulti-[A]{.underline}gency [L]{.underline}aboratory
[A]{.underline}nalytical [P]{.underline}rotocols (Manual)

MARSAME [M]{.underline}ulti-[A]{.underline}gency [R]{.underline}adiation
[S]{.underline}urvey and [A]{.underline}ssessment of
[M]{.underline}aterials and [E]{.underline}quipment (Manual)

MARSSIM [M]{.underline}ulti-[A]{.underline}gency [S]{.underline}urvey
and [S]{.underline}ite [I]{.underline}nvestigation [M]{.underline}anual

M&E [M]{.underline}aterials and [E]{.underline}quipment

MC True Background Distribution

MCE Random Sample of Background

MDC [M]{.underline}inimum [D]{.underline}etectable
[C]{.underline}oncentration

MQC [M]{.underline}easurement [Q]{.underline}uality
Un[c]{.underline}ertainty (also [M]{.underline}inimum
[Q]{.underline}uantifiable [C]{.underline}oncentrations)

MQO [M]{.underline}easurement [Q]{.underline}uality
[O]{.underline}bjective(s)

N The Sample Size ([N]{.underline} measurements, for instance)

NCRP [N]{.underline}ational [C]{.underline}ouncil on
[R]{.underline}adiation [P]{.underline}rotection and Measurements

NHSRC [N]{.underline}ational [H]{.underline}omeland
[S]{.underline}ecurity [R]{.underline}esearch [C]{.underline}enter

NIST [N]{.underline}ational [I]{.underline}nstitute of
[S]{.underline}tandards and [T]{.underline}echnology

NRC [N]{.underline}uclear [R]{.underline}egulatory
[C]{.underline}ommission (U.S. NRC)

NUREG NRC [NU]{.underline}clear [REG]{.underline}ulatory Guide (U.S.
NRC)

OAR [O]{.underline}ffice of [A]{.underline}ir and
[R]{.underline}adiation (U.S. EPA/OAR)

ORIA [O]{.underline}ffice of [R]{.underline}adiation and
[I]{.underline}ndoor [A]{.underline}ir (U.S. EPA/OAR/ORIA)

PAG [P]{.underline}rotective [A]{.underline}ction [G]{.underline}uide

pdf [P]{.underline}ortable [D]{.underline}ocument [F]{.underline}ormat

q critical value for statistical tests

QA [Q]{.underline}uality [A]{.underline}ssurance

QC [Q]{.underline}uality [C]{.underline}ontrol

QA/QC [Q]{.underline}uality
[A]{.underline}ssurance/[Q]{.underline}uality [C]{.underline}ontrol

R~b~ Mean Background Count Rate

RAC [R]{.underline}adiation [A]{.underline}dvisory
[C]{.underline}ommittee (U.S. EPA/SAB/RAC)

rev [Rev]{.underline}ision

SAB [S]{.underline}cience [A]{.underline}dvisory [B]{.underline}oard
(U.S. EPA/SAB)

Standard deviation

~M~ Standard Deviation of Measurement Error

~S~ Standard Deviation of Sampling Distribution

S~c~ Critical Value

SI [I]{.underline}nternational [S]{.underline}ystem of Units (from NIST,
as defined by the General Conference of Weights & Measures in 1960)

SOP Standard Operating Procedure

Φ~mr~ The relative upper bound of the estimated measurement method
uncertainty μ~mr~,

t~B~ Background Time

t~S~ Sample Time

TSCA Toxic Substances Control Act

Type I Type I error is rejecting the null hypothesis when it is true

Type II Type II error is failing to reject the null hypothesis when it
is false

μ~mr~ Estimated Measurement Method Uncertainty

Uncertainty (e.g., ~MR~)

UBGR [U]{.underline}pper [B]{.underline}ound of the [G]{.underline}ray
[R]{.underline}egion

UCL [U]{.underline}pper [C]{.underline}ontrol [L]{.underline}imit

US [U]{.underline}nited [S]{.underline}tates

W~r~ Adjusted Reference Measurement (WRS test)

W~s~ Sum of the Ranks of the Sample Measurements (WRS test)

WRS [W]{.underline}ilcoxon [R]{.underline}ank [S]{.underline}um Test

y0 Estimate of Zero Order Output Quantity; also Minimum Detectible
Concentration

Z Critical Regions (e.g., Z ~1-~ , or Z ~1\ -\ ,~ that is, quantile of
the standard normal distribution)

**APPENDIX C -- TYPOS AND CORRECTIONS**

xxix line 504 power?

522 delete one (

xxxi 561 delete one )

567 delete one (

xxxiv 671 Technetium (sp.)

xxxv 676 delete (duplicates 675)

1-3 80 change "activity concentrations" to "area activity" or leave as
is but change "Bq/m^2^" to "Bq/m^3^" and add "and area activity
(Bq/m^2^)

3-9 194 non-radionuclide-specific (insert dash)

4-5 Figure 4.1a replace second "Large" by "Much Larger"

Figure 4b. replace second "Small" by "Equally Small or Smaller"

5-21 523 value in denominator should be 0.4176 (see line 527)

527 plus should be behind square root of 87

5-53 1148 delete 2^nd^ period

6-6 142 insert "to" behind "likely"

6-11 280 insert "that" behind "determine"

6-13 329 insert "that" behind "demonstrate"

6-23 474 and 482 critical value in symbols table is not in italics
(italicized k is coverage factor)

7-10 210 Tl-208 should be beta/gamma, not just beta, with gamma-ray
energy in next column

B-6 151 maximize, not minimize

D-9 219 what does "varies" mean?

D-36 849 for LS spectrometer, insert (alpha) on first line of column 2
and (gamma) for the HPGE and NaI detectors

F-1 26 delete (FRER)

[^1]: Mr. Napier was unable to attend the face-to-face meeting of
    October 29-31, 2007 and the conference call of March 10, 2008.

[^2]: The parenthetical numbers identify recommendations in response to
    the charge questions.