Strategic Assessment of 

Sampling Resources

Report
2

Contents

Executive Summary.......................................................................................................................................5

MASTER TABLE............................................................................................................................................10

Introduction ................................................................................................................................................12

Background............................................................................................................................................12

Design and Structure of the Evaluation...............................................................................................13

Scope ......................................................................................................................................................14

Data.........................................................................................................................................................14

Participants............................................................................................................................................14

Phase 1: Describe Current Sampling Projects ...................................................................................15

Approach and Methodology .................................................................................................................15

Analysis ..................................................................................................................................................15

Results....................................................................................................................................................15

Recommendations.................................................................................................................................15

Phase 2: Develop Weighted Criteria against which to Evaluate Sampling Projects ..................16

Approach and Methodology .................................................................................................................16

Analysis ..................................................................................................................................................16

Results....................................................................................................................................................18

Recommendations.................................................................................................................................22

Phase 3: Assess Whether Each Sampling Project, as Implemented, Can Provide the Intended 

Information and Benefits ......................................................................................................................23

Approach and Methodology .................................................................................................................23

Analysis ..................................................................................................................................................29

Results....................................................................................................................................................29

Recommendations.................................................................................................................................33

Phase 4: For On-Going Sampling Projects, Determine Whether FSIS Has Used the Results for 

Agency Decisions.....................................................................................................................................36

Approach and Methodology .................................................................................................................36
 
4

List of Figures

Figure 1. Criteria Ranking and Weights .............................................................................................................19

Figure 2. Unweighted and Weighted Scores......................................................................................................21

Figure C1. Rank Order Centroid Example with 5 Ranked Items......................................................................50

List of Tables

Table 1: Summary of Sampling Projects Ranked by Unweighted and Weighted Score .............................20

Table 2: Designated Standards for the Maximum Margin of Error Calculations by Pathogen Rate ..........25

Table 3: Tool to estimate the number of samples required for a given pathogen rate and a set maximum 

margin of error, assuming a 95% confidence interval................................................................................27

Table 4: Upper bound number of samples required calculations by pathogen rate and confidence........28

Table 5: Parameters for a standard maximum margin of error of a confidence interval for an estimate of 

a pathogen rate...........................................................................................................................................30

Table 6: Subset of Table 4 (Upper bound sample size calculations by pathogen rate and confidence) ....30

Table 7: Indirect Inspection Time Estimates for Large Establishments......................................................39

Table 8: Summary of Cost by Laboratory Analysis and Sample Collection......................................................40

Table B1: Description of Ongoing Sampling Projects..........................................................................................43

Table D1: Detailed Responses to Benefits Questions including Weighted/Unweighted Scores by 

Project.......................................................................................................................................................................52

Table E1: Estimated Number of Samples Required for Projects with Published Pathogen Rates...............56

Table F1: Analyses of data by project...................................................................................................................57

Table G1: Estimate of Costs by Sampling Project................................................................................................68

Table H1: Detailed Master Table Summary by Individual Project Scatter Plot...............................................73

Table H2: Detailed Master Table Summary by Individual Project Tablix.........................................................74

List of Boxes

Box 1: Criteria for Determining the Potential Benefits for Individual Sampling Projects..............................17

Box 2: Overview of Statistical Terms ....................................................................................................................24

Box 3: Questions to Help Consider whether Sampling Project Designs are Adequate..................................35
5

Executive Summary

The U.S. Department of Agriculture’s Food Safety and Inspection Service (FSIS) leverages robust 

sampling projects to verify the safety of products regulated by the Agency. In fiscal year (FY) 2018, the 

year this evaluation began, the Agency collected over 120,000 samples for laboratory microbiological

and chemical residue analysis through these projects. FSIS reported over 2,000,000 different analytes 

from these results and collected and analyzed almost 4,000 samples for further examination of 

veterinary diseases.

As a science-based agency, FSIS uses data to inform decision making and drive continuous improvement 

of processes. To maximize the efficiency, effectiveness, and value of these sampling projects, the 

Agency undertook a systematic evaluation to fully account for and prioritize resources. The Strategic 

Assessment of Sampling Resources (SASR) evaluation team (or SASR workgroup) within FSIS was formed

to design and conduct this evaluation. The SASR Workgroup incorporated broad, cross-cutting 

participation from multiple program areas, including the Agency’s pathogen and chemical residue 

workgroups. FSIS supplemented its expertise with contracted work to conduct and develop portions of 

this strategic evaluation. The evaluation was conducted from September 2017 to May 2019. The 

assessment compiles information from a variety of different sources, including Public Health 

Information System (PHIS) sampling data, Agency reports, FSIS notices and directives, and relevant 

Federal Register notices, among other sources.

The underlying premise guiding the SASR workgroup through the evaluation was that FSIS sampling only 

fulfills its purpose when the data it generates is used by the Agency. Relying on that guiding principle, 

the SASR workgroup developed a framework to assess whether data generated under each of the 

Agency’s sampling projects are analyzed, and if the results of those analyses are factored into the 

Agency’s decision-making. 

The workgroup developed a multiphase approach, with the sixth and final phase earmarked for future 

development:

• Phase 1: identify and describe all current sampling projects and the reason behind each. 

• Phase 2: develop weighted categories and criteria to use for scoring and ranking the potential 

benefits of each project. 

• Phase 3: determine whether each sampling project, as implemented, could satisfy the stated 

policy objective or its intended purpose. 

• Phase 4: assess whether data from ongoing sampling projects is being used by the Agency as 

originally intended.

• Phase 5: conduct a cost assessment across all sampling projects. 

• Phase 6 (future phase): conduct a semi-quantitative evaluation, based on work from Phases 1-5,

to provide rankings for current and future sampling projects

This report details the outcomes and results of Phases 1-5 and includes a master table of broad sampling 

programs and initiatives. More detailed results from individual projects are provided in Appendix G: 

Details of Costs Associated with Sampling Projects.

Based on the results, the SASR workgroup identified nine major findings and made recommendations to 

address them, including some related to internal FSIS workgroups. These recommendations will increase 

the Agency’s efficiency and improve allocation of resources. These findings and associated 

recommendations can be separated into 2 major categories: Process-Oriented and Statistically-Oriented.
6

Process Findings and Recommendations

1. Process Improvement Findings

A. The Sampling Coordination Committee (SCC) is underutilized. The SCC was formed to centralize 

responsibility for the organizational aspects of sampling project development and maintenance. 

However, the SCC could be better incorporated into the management review process for the 

various pathogen and chemical residue workgroups. Also, the current SCC Annual Sampling Plan 

Change Request Form could be improved upon by collecting additional information —potential 

project benefits; statistical sampling needs and plans; and how the Agency will analyze and use 

the data—to assess proposed sampling projects.

SASR Actions: The workgroup reviewed the SCC form and developed tools to address 

deficiencies. 

Recommendations:

1.1: The SCC should revise the current Annual Sampling Plan Change Request Form to 

incorporate the following:

a) the benefit questions established by this evaluation (page 22);

b) a question to determine whether the information can be obtained by updating an 

existing sampling project (page 37);

c) more detailed statistical documentation regarding the number of samples required and 

sample allocation methodology (page 33); and

d) a schedule for routine evaluation—with a formal analysis plan— for each sampling 

project to determine whether the results of the project are being used to inform FSIS

policy (page 37).

1.2: New sampling project sponsors and the SCC should ensure that a sampling project’s design 

has been optimized. Further, they should ensure that the information gained from the sampling

will be used before any decision to institute a sampling project, and consider the overall costs 

associated with the sampling plan (page 41).

B. FSIS laboratories do not have a centralized and robust cost projection tracking system for all 

laboratory related costs. 

Recommendation:

1.3: FSIS should utilize Office of Public Health Science (OPHS) resource subject matter experts to 

create and continuously refine a repository that projects and track resource requirements by lab 

and project (page 40).

2. Sampling Project Sunsetting Finding

A. The Agency does not have a consistent, formal process to assess when a project or portions of 

a project should “sunset”. 

Recommendation:

2.1: FSIS should review existing exploratory/baseline sampling and make determinations to 

either discontinue sampling or incorporate the exploratory portion of the sampling into a 

routine sampling project (page 37).

2.2: For projects developed after these recommendations are implemented, the Agency should 

ensure there is a specified sunset date for each project (page 37).

■ 
7

3. Sampling Project Inventory Finding

A. There is no single, comprehensive inventory of sampling projects that includes how they are 

designed and why they are initiated and sustained. In 2011, the Agency created a complete 

inventory of sampling projects, but it has not been updated since the original 2011 sampling 

plan1.

SASR Actions: The workgroup created a detailed inventory of all ongoing sampling projects that

includes the name of each project, the policy objective or intended purpose, and how the 

individual sample results and aggregate data are being used to inform Agency policy decisions.

Recommendation: 

3.1: The SCC should annually review the inventory through the routine development of the 

Annual Sampling Plan and propose if the inventory should be internally or publicly available

(page 15).

4. Implementation of Weighted Criteria Finding

A. The semi-quantitative approach, developed by the SASR workgroup, is an appropriate and 

useful method to evaluate the benefits of each sampling program.

SASR Actions: The workgroup developed weighted criteria and ranked sampling projects to 

ensure that overall Agency needs are considered when deciding when to initiate and sunset 

sampling programs. 

Recommendations:

4.1: All sampling projects that ranked below 0.30 on the benefit score (see Table 1: Summary of 

Sampling Projects Ranked by Unweighted and Weighted Score) should be evaluated by the 

various Data Coordination Committee (DCC) pathogen/chemical workgroups. Results of the 

evaluation should be reviewed through the Enterprise Governance process. Based on the 

findings presented, continuation of any project below 0.3 should be explicitly endorsed by the 

Enterprise Steering Board (ESB) (page 22).

4.2: Moving forward, the sampling project proposers and SCC should adopt the weighted

benefits criteria from this report to evaluate the potential benefits of proposed new or revised 

sampling projects (page 22).

4.3: The sampling project proposers and SCC should also ensure that a sampling project’s design 

has been optimized, consider how the information from the sampling will be used by the 

Agency, and consider the overall costs associated with the sampling plan (page 41).

5. Outreach and Communication Finding

A. Finding: SASR workgroup recommendations will change processes for sampling projects and 

could cause confusion during implementation.

Recommendation: 

5.1: The SASR workgroup will coordinate with Office of Public Affairs and Consumer Education

(OPACE) to communicate findings and recommendations to FSIS employees, including a 

description of the new process and tools that will be available. This new information should be 

placed on the new SCC Sampling Change Request Form (see Recommendation 3a), currently 

located on the SCC SharePoint site, to be accessible for applicable parties.

 

1 Report on the Food Safety and Inspection Services’ Microbiological and Residue Sampling Projects. 2011. 

https://www.fsis.usda.gov/wps/wcm/connect/0816b926-c7ee-4c24-9222-34ac674ec047/FSIS_Sampling_Project 

s_Report.pdf?MOD=AJPERES
8

Statistical-Oriented Findings and Recommendations

6. Alignment of Statistical and Policy Goals Finding

A. Agency sampling projects have differing levels of utility and cost-effectiveness, and some 

sampling projects could be optimized by altering the number of samples collected and analyzed. 

SASR Actions: The workgroup collaborated with MITRE to develop a tool that provides a variety 

of statistically-based calculations as a starting point for optimizing the number of samples that 

should be collected and analyzed for each sampling project. 

Recommendations: 

6.1: When a request is submitted for a new or revised sampling project, the submitter should 

evaluate the project to optimize sample allocation. To meet this recommendation, the 

workgroups should (page 33):

a) Using the tool as a starting point, determine the appropriate number of samples to 

analyze to ensure that any intended benefits (e.g. maximum margin of error, upper 

bound claims, or comparison of rates) can be realized given the statistical limitations of 

its design.

b) Determine the minimum number of samples required to meet Agency policy goals (e.g., 

establishment categorization).

c) Determine the number of samples required to realize both the statistical and policy 

benefits annually, and document and include it in the workgroup submission to the FSIS 

Annual Sampling Plan. 

7. Sampling for Products with Very Low Pathogen Rates Finding

A. For domestic sampling projects with very low positive rates it is not feasible to collect and 

analyze enough samples to produce reliable estimates.

SASR Actions: The workgroup used the MITRE tool ‘upper bound estimator’ to develop a table 

to help guide decision making.

Recommendation:

7.1: FSIS should consider the statistical claims that can be made based on the number of 

samples collected and analyzed [if no positive samples are detected] (page 34).

7.2: FSIS should clearly document the reasoning and the potential statistical claims that are 

associated with each sampling project (page 34).

8. Prevalence Estimation Finding

A. FSIS has not established clear and consistent standards for when sampling data can be used to 

estimate national prevalence. The ability to estimate pathogen prevalence in establishments is 

not solely a matter of the number of samples that are included in the analysis. The margin of 

error around the estimate decreases as the number of samples increases.

SASR Actions: The workgroup reviewed FSIS documentation (reports, analyses) regarding 

prevalence estimation to identify aspects for improvement.

Recommendation:

8.1: FSIS should develop clear standards for determining whether a prevalence estimate can be 

calculated for each FSIS in plant sampling projects, including identifying whether there is a 

maximum margin of error for calling an estimate a “prevalence.” (page 34)

8.2: For all projects, FSIS should clearly document the reasons a prevalence estimate can or 

cannot be calculated (page 34).
9

9. Sampling Project Specific Results Findings

A. Sampling of imports at reinspection serves different purposes and has different statistical 

design challenges than domestic sampling. To obtain a statistically representative sample, 

significantly more import samples would need to be taken, the PHIS import module would 

require a rework and requiring additional work with international stakeholders.

Recommendation:

9.1: FSIS should conduct a separate evaluation of import sampling to maximize the benefits it 

provides to the Agency (page 34).

B. The egg product workgroup evaluated the sampling model and suggested an updated 

methodology. However, at the time this evaluation was conducted, the Agency had not yet 

made a final decision on whether and how to modify egg product sampling.

Recommendation: 

9.2: FSIS should complete review of the egg product sampling proposal and implement any 

changes by FY 2020 (page 34).
MASTER TABLE

The SASR workgroup evaluated each sampling project using the criteria and categorized costs of the project. The costs have been categorized 

into low, medium and high categories to simplify the cost/benefit ratio by grouping. Detailed results of that evaluation are provided in Appendix 

H. The master table below provides an overall summary of the results of the evaluation of the sampling programs that were evaluated. It

provides both a graphical and tabular representation of the benefit score and cost per sample for the sampling projects rolled up to their 

unifying groupings. 

1.00 

0.80 

I!' 

~ 0.60 

1 

g> 0.40 < 

0.20 

0.00 

Projects 

Benefit Score 

S Category 

Samples 

- 1.0 

0.8 

I!' 

~ 0.6 

~ 

~ 0.4 

0.2 

0.0 

RTEPROD Sampling 

RTU>ROD Samolina 

• 

• .9 '0 

~ = "' 'r 

- ~ M 

Import • Sampling 

llow 

Beef Sarrl)lina Poultrv Verification 

I • 

• .9 '0 

~ = "' 'r ~ '0 ~ = "' 'r 

- ~ M - ~ M 

CostCat o 

' Ve ry low Volume Sampling 

2 Medium 

RLM 

For-Cause Samolina Eaa Product Sam lina RLM Res.idue 

• • • • I • 

• .9 '0 

~ = "' 'r ~ '0 ~ = "' 'r • .9 '0 ~ = "' 'r ~ '0 ~ = "' 'r 

- ~ M - ~ M - ~ M - ~ M 

Residue Exploratory Pork 

0.23 

4491'. 

Follow-uo s.amolina lmoort Sam lina Exoloratorv Pol'X 

I • I I 

• .9 '0 

~ = "' 'r • .9 '0 ~ = "' 'r • .9 '0 ~ = "' 'r 

- ~ M - ~ M - ~ M 

Residue 

Import • Sampling 

O• the r

3 High 

Exploratory Poultry 

4 

0.23 

2 

780 

vnln atorv Poultrv 

• • 

• '0 ! 1 • .9 ~ .9 

- ~ -

Other 

0.16 

Othe, 

• 

:I ! '0 

~ = "' 'r 

~ M 

Ve ry Low Volume 

Sampling 

0.15 

Very low Volume 

Sampling 

• .9 '0 

~ = "' 'r 

- ~ M 
11

• For each of the grey rows (i.e. Projects and Samples), the darker the color the higher the value. On the blue row (Benefits score), the 

darker the color, the higher the benefit score, while on green row ($ Category) row, the darker the color, the lower the average cost per 

sample to run the projects. Different color schemes are used to improve readability between the rows and between the points in the 

scatter plots.

• The values in the Projects row indicate the number of projects included in the grouping.

• The $ Category is the average cost of sampling for all projects within the grouping. 

• The Samples row contains the combined number of samples for the projects that were allocated in the FY 2019 Annual Sampling Plan.

• The top scatter plot relates the average benefit score to the average total cost per sample rolled up to the grouping level. The size of the 

points in the top scatter plot represent the cumulative number of samples for that grouping. The larger the point, the more samples are 

included in that grouping. Not all project names are listed on the scatter plot. 

• The bottom scatter plot relates the cost per sample to the benefit score for that sampling project.

• The points on the bottom scatter plot represent the individual projects. Note: there may be overlap with several projects in the grouping 

as projects may have identical benefits scores and cost per sample, resulting in fewer points represented.
12

Introduction

In 2016, the Food Safety and Inspection Service (FSIS) formed the Evaluation Working Group (EWG), 

which reports to the Enterprise Steering Board (ESB), to provide a more formal mechanism to prioritize, 

coordinate, integrate, and collaborate on FSIS evaluation activities. In 2017, the EWG developed and 

began work on a prioritized list of all potential evaluations that could be conducted by the Agency. One 

of these priorities was a strategic evaluation of all FSIS sampling projects, which included developing 

evaluation criteria. This effort included a retrospective evaluation of existing projects and development 

of a framework and tools to prospectively evaluate new sampling projects. To design and conduct this 

evaluation, a Strategic Assessment of Sampling Resources (SASR) evaluation team was formed.

This evaluation is consistent with FSIS’ fiscal year (FY) 2017-2021 Strategic Plan, which has a specific 

objective under Goal 1 of strengthening sampling projects. This report summarizes the approach and 

methods used, as well as results, findings, and recommendations from the evaluation.

Background

FSIS conducts robust sampling projects for the products it regulates, with laboratory testing of samples 

conducted by FSIS field service laboratories. FSIS sampling is an important component of how FSIS

verifies that establishments have addressed food safety hazards and are producing safe, unadulterated, 

and properly labeled product.

In general, for this document, broad FSIS sampling efforts related to a chemical residue or specific 

pathogen, such as Salmonella, is referred to as a “sampling program,” whereas an individual sampling 

effort, such as HC_CH_COM01 for Salmonella, is referred to as a “sampling project”.

FSIS collects samples at domestic, federally-inspected establishments, in commerce (for example, at 

retail establishments), and at FSIS-regulated import houses (i-Houses). FSIS also collects samples as part 

of outbreak investigations and in response to consumer complaints. Samples are collected and sent to 

one of FSIS’ laboratories, which conduct microbiological analysis, chemical residue analysis, or other 

analyses, such as pathology or speciation. In addition, FSIS inspectors conduct many chemical residue 

screening tests in the establishment, such as Kidney Inhibition Swab (KIS) tests. However, this evaluation 

focuses on samples from projects that are analyzed in FSIS laboratories; in-establishment screening tests 

are not evaluated here.

For microbiological sampling, FSIS currently performs analyses for Salmonella, Campylobacter, Listeria 

monocytogenes (Lm), Indicator Organisms, E. coli O157:H7 and non O157 Shiga toxin-producing 

Escherichia coli (STEC) for various products including raw beef, poultry, pork, and Siluriformes, as well as 

ready-to-eat (RTE) products. For a detailed description of most of FSIS’ sampling projects, please see the 

Agency’s 2011 Report on the Food Safety and Inspection Service‘s Microbiological and Residue Sampling 

Projects.

FSIS’ Sampling Coordination Committee (SCC) was established in 2016 to provide a forum through which 

sampling strategies and program recommendations are discussed prior to being presented, via the Data 

Coordination Committee (DCC), to the Executive Governance Boards and the Management Council for 

official authorization to implement or modify sampling programs. The SCC has the authority to identify 

and review sampling matters and to make recommendations for sampling actions that support the 

mission of FSIS. The SCC maintains a Sampling Plan Change Request Form, which was developed to 
13

ensure that all pertinent questions were addressed before approving a sampling project. The Form also

serves as a permanent record of changes to sampling projects; approved forms are stored on the SCC 

SharePoint site. When recommending changes to the collective Agency sampling program via the FSIS 

Annual Sampling Plan or individual sampling projects, the SCC considers the sampling standardization, 

organization resource commitments, sampling portfolio management, overlap and interagency process 

interaction.

Purpose of Evaluation

Historically, each FSIS sampling project has been designed independently, with unique purposes and 

goals. Sampling projects are resource intensive and, therefore, FSIS determined that an overall 

evaluation of FSIS’ sampling projects was necessary to maximize the efficiency, effectiveness and value 

of those projects. The purpose of this evaluation, therefore, was 1) to develop a method to evaluate 

FSIS’ ongoing and future sampling programs; and 2) use that method to assess the allocation of FSIS’ 

sampling resources across the different sampling projects to ensure that the Agency is strategically 

utilizing sampling resources.

Design and Structure of the Evaluation

Given this is a novel, holistic assessment of the Agency’s sampling projects, the SASR workgroup first 

needed to develop a framework—that is, an approach and method—for conducting the evaluation. An 

early consensus among workgroup members was that an underlying premise of this evaluation is that 

FSIS sampling is only useful if the data generated are used by the Agency. Thus, the workgroup sought to 

develop a framework that included an assessment of whether data generated under the Agency’s 

sampling projects are analyzed, and whether the results of the analysis are factored into the Agency’s 

decision-making.

 

With that in mind, the workgroup developed the framework—through an iterative process that included 

multiple consultations with FSIS subject matter experts (SMEs) and executives from across the Agency—

and identified the necessary pieces of information to properly assess the Agency’s sampling efforts. That 

framework consists of the following five phases, organized according to different aspects of the 

evaluation: 

• Need: A robust inventory of all sampling projects.

o Phase 1: Describe current sampling projects.

• Need: A systematic, equitable way to assess the relative strengths and weaknesses of each 

sampling project based on the information that a sampling project can provide to the Agency, 

and the importance of that information.

o Phase 2: Develop weighted criteria against which to evaluate sampling projects

• Need: A way to determine whether the number of samples collected and analyzed for each 

sampling project achieves the stated Agency goal of the project. 

o Phase 3: Assess whether each sampling project, as implemented, can provide the 

intended information/benefits

• Need: A systematic, equitable way to assess whether the results from each sampling project are 

being analyzed, and whether those results are used to inform FSIS policy decisions.

o Phase 4: For on-going sampling projects, determine whether FSIS has used the results

for Agency decisions

• Need: A systematic, equitable way to assess the relative costs of each sampling project.

o Phase 5: Assessment of costs across sampling projects
14

Scope

This evaluation included all active routine and inspector-generated sampling projects, as well as followup/for-cause sampling the Agency performs and analyses in its laboratory (e.g. follow-up sampling in 

response to a prior positive sampling result). Special Programs and Outbreak sampling projects were 

excluded as they are non-routine and not continuously running projects. In establishment screening KIS 

tests, which are conducted by in-plant personnel, were also not included in this evaluation. 

Data

This analysis utilized a listing of all current sampling projects from the FSIS Public Health Information 

System (PHIS), descriptions from Federal Register Notices, FSIS Directives and Notices that indicate why 

each sampling project was initiated, information collected from SMEs on how FSIS used aggregate 

analyses from each sampling project, and cost information supplied by FSIS laboratory management, 

Office of Field Operations (OFO), budget staff, and economists. 

Participants

The SASR workgroup includes staff from a broad cross-section of the agency, including the Office of the 

Chief Financial Officer (OCFO), Office of International Coordination (OIC), Office of Field Operations 

(OFO), Office of Investigation, Enforcement, and Audit (OIEA), Office of Planning, Analysis and Risk 

Management (OPARM), Office of Policy and Program Development (OPPD), and the Office of Public 

Health Science (OPHS).

The SASR workgroup reached out to FSIS SMEs to help describe and assess individual sampling projects, 

provide cost estimates, and help select appropriate statistics for evaluating whether a sampling project 

is designed in a way to meet Agency and policy goals. Participants predominantly acted through one or 

more of the workgroups, or as SMEs on various topics, as identified below:

• Pathogen and Chemical Residue Workgroups (PWG; includes commodities and residues)

o Salmonella and Campylobacter Coordination Group (SCCG)

o Ready-to-Eat (RTE) Workgroup

o Shiga toxin-producing Escherichia coli (STEC) Workgroup 

o Egg Products

o Residues

• OCFO Budget Analysts

• Laboratory personnel

• FSIS statisticians

• FSIS economists

• MITRE Consultants

A full Participant List is presented in Appendix A: Evaluation Participants.
15

Phase 1: Describe Current Sampling Projects

Approach and Methodology

In Phase 1, the SASR workgroup compiled a complete list of all existing FSIS sampling projects captured 

in PHIS and the Laboratory Information Management System (LIMS). Included in this data compilation 

was the project code, project name, and the purpose of sampling. To understand the purpose of each 

sampling project, the workgroup and SMEs reviewed historic Annual Sampling Plans, Annual Sampling 

Catalog, all relevant Federal Register Notices (FRNs), FSIS Directives and Notices related to Agency 

sampling, as well as other relevant FSIS Notices and Directives, internal FSIS documents, including the 

SCC Sampling Plan Change Request Form, and external sources to identify rationale for initiating each

sampling project. 

Analysis

The SASR workgroup conducted a side by side evaluation with the Annual Sampling Plan to determine 

whether the plan provided enough information to serve as a full inventory.

Results

Complete results for all current sampling projects, including the project code, the name of the sampling 

project, and the purpose of the sampling as described, when available, in the FRN or FSIS Directive or

Notice announcing its initiation, are included in Appendix B: Description of Ongoing Sampling Projects.

For some sampling projects, FSIS uses a different project code for samples of the same productpathogen pair that are sent to different laboratories for analysis. For the purposes of summarizing the 

data in this report, and to avoid double-counting, when there were different project codes for the same 

product-pathogen pair being sent to different laboratories, those project codes were assessed together 

and are only counted once. 

The side by side comparison of the Annual Sampling Plan with the inventory that was created during this

phase identified several gaps, including that the sampling plan covered a more limited scope of projects 

and there were no details concerning the historic policy background for initiating all sampling programs. 

Recommendations

The SCC should annually review the inventory through the routine development of the Annual Sampling 

Plan and propose if the inventory should be internally or publicly available.
16

Phase 2: Develop Weighted Criteria against which to Evaluate 

Sampling Projects

Approach and Methodology

In Phase 2, the SASR workgroup developed categories and criteria to use for a qualitative evaluation of 

the potential benefits of each sampling project. Evaluation of each of the 157 sampling projects

identified in Phase 1 is based on the extent to which the project contributes to the Agency’s mission and 

goals. An Excel spreadsheet was created to track scoring. This qualitative evaluation included the 

following four steps:

1. Establish the criteria;

2. Assign each sampling program a score of either one or zero, according to whether it could or 

could not potentially provide the information, or benefit, for the given criteria; 

3. Determine criteria weights using within-Agency expert elicitation and rank ordering; and

4. Compute project total score as the weighted sum of the benefit scores.

Analysis

Establish the Criteria

To determine the potential benefits of any given sampling project, the SASR workgroup identified four 

broad categories of ways that the Agency uses data from its sampling projects in decision-making: 1) to 

assess individual establishments or countries; 2) to conduct investigations; 3) to modernize regulations; 

and 4) to track Agency progress. 

The workgroup then created a series of criteria, in the form of questions, under each of the four 

categories, for a total of 18 questions to gather information on what benefits could potentially come 

from the different sampling projects (see Box 1 for the 18 criteria questions). For each sampling project, 

the questions could be asked to determine whether data from the sampling project could provide the 

given benefit. If it could, a score of one would be entered into an Excel spreadsheet; if it could not, a 

score of zero would be entered into an Excel spreadsheet. The SASR workgroup met several times to 

discuss the scores for each of the projects and through these robust discussions came to a consensus 

over which values were appropriate for the given project/question. The scores for each question—zero 

or one—could then be summed for each sampling project to provide an overall rating or ‘score’ for the 

sampling project. That overall score reflects the amount of information, or potential benefits, that each 

sampling project provides to FSIS for its decisions. 

As a means of peer review, the SASR workgroup had SMEs peer review the criteria and test their 

applicability by evaluating two different FSIS sampling projects—the STEC sampling projects and 

pathology sampling projects. The peer review was to ensure that the criteria were adequate and would 

be applicable to FSIS’ sampling projects. The questions were refined based on feedback received. 

Additional SMEs were then enlisted to evaluate all ongoing FSIS sampling projects against those refined 

criteria. 
17

Determine Criteria Weights

As an initial assessment, all criteria were given the same weight, such that each time the SMEs indicated 

that a sampling project, as it was designed at the time of the assessment, could potentially provide the 

specific information described in the criterion question, it was given a score of one. The overall score for 

potential benefits for each sampling project was then calculated by summing the number of positive 

“yes” responses across all criteria questions.

1. Assess Individual Establishments or Countries

a. Food Safety Related:

i. Is the sampling project testing for an adulterant that is likely to occur, including residues with 

tolerances?

ii. Are the sampling results used to direct a Food Safety Assessment for an establishment or 

supplier?

iii. Do the sampling results indicate whether an establishment is meeting FSIS’ performance 

standards, such as Salmonella performance standards for poultry? 

iv. Does the sampling project contribute to other surveillance, verification or follow-up activities 

at specific establishments, in commerce facilities or countries? 

v. Does the sampling project test for food defense-related threat agents?

b. Non-Food Safety Related:

i. Do the sampling results provide inspection personnel information on animal diseases that are 

diseases for which a carcass would be condemned?

ii. Does the sampling project identify misbranding, such as a non-compliance related to economic 

adulteration?

5. Conduct Investigations

a. Is the sampling project conducted to investigate an outbreak to link contamination in FSIS-regulated

regulated products to clinical illnesses?

b.Is the sampling project conducted for follow-up investigations in special circumstances other than 

outbreak investigations?

6. Modernize Agency Policies and Regulations

a. Are the sampling results being analyzed to inform the development of FSIS regulations or policies? 

b.Are the sampling results used to determine the hazard or risk of an emerging pathogen, chemical or other 

agent in an FSIS-regulated product?

c. Do the sampling results provide information about a hazard (microbial or chemical) in a new or different 

product?

7. Track Agency Progress, Conduct National Surveillance, and Establish/Support Priorities

a. Does FSIS use the results of the sampling project to track its progress in advancing Agency’s the Agency

towards a specific performance measure from its Strategic or Annual Plan?

b.Is the sampling project conducted to provide information, at the national level, on the prevalence of 

antimicrobial anti-microbial resistance?

c. Do the results provide information on the presence of bacterial types or strains that typically are 

associated with human illnesses?

d.Does the sampling project provide information, at the national level, on the prevalence or percent 

positive of a microbe, chemical or other agent of concern?

e. Is the sampling project conducted to support data collection efforts for another agency, or as a 

requirement of an MOU?

f. Do the sampling results provide inspection personnel information on animal diseases that are diseases for 

which FSIS conducts surveillance from an animal health perspective?

Box 1: Criteria for Determining the Potential Benefits for Individual Sampling Projects
18

However, after review of the overall scores and discussion amongst the SASR workgroup, as well as with 

FSIS SMEs and executives, the consensus of the SASR workgroup was that all potential benefits, or 

information, are not equally important to the Agency. For example, information on the presence of an 

adulterant is considered more important—or of greater benefit—than information for a Memorandum 

of Understanding (MOU). Therefore, the workgroup attached weights to the benefits questions. 

To do so, the SASR workgroup polled its members as well as Agency SMEs to determine the value they 

would attach to each benefit question, ranking them in order of importance from highest (1) to lowest 

(18). Next, the workgroup reviewed the results of its polling and developed a consensus ranking for each 

benefit criteria. MITRE then explored different methods to convert the rankings to weights. (See 

Appendix C: ROC Method for more information on the ROC method.) The rank-order centroid (ROC) 

method was selected because it provided more differentiation among the criterions than other methods 

examined. The ROC methodology uses a standard competitive ranking process; that is, in the case of a 

tie, the ranking would skip (e.g., 1, 1, 3 ...) to preserve the same ranking scale across program areas. The 

final criteria ranking is computed from the median program area response. 

As a final review, FSIS SMEs examined the overall weighted scores for the different sampling projects to 

assess whether the rankings based on those scores reflected the importance of the sampling data to 

FSIS’ mission. Minor changes were made in the weighting to better align the scores with FSIS’ mission. 

The result of the weighting and minor adjustments was that the final scores and rankings based on those 

scores more accurately reflected the relative benefits each sampling project provides. 

Results

Rank and Weight Criteria

The ROC weight, using the standard competitive ranking process, is shown in Error! Reference source 

not found.. The final weighting for each criteria question is listed in the far-right column, and the 

number for the criteria (from Box 1) are listed from highest to lowest weighting. As can be seen in Figure 

1, Criteria 1.a.i, which is whether the sampling programs tests for adulterants, is clearly ranked as the 

Agency’s top priority and has the highest weight. As discussed above, the rank corresponds to the value 

assigned to each question (on a scale of 1-18) based workgroup and Agency’s consensus.
19

Figure 1. Criteria Ranking and Weights. The criteria numbers correspond to the criteria in Box 1 

and are color-coded for the category of potential benefits from the information: blue, to assess 

individual establishments or countries; yellow, to conduct investigations; green, to modernize 

Agency policies; and gold, to track Agency progress.

Score Projects Across Criteria and Compute Total Scores

SMEs reviewed each of the sampling projects and assigned the criteria a score of 1 if met by the 

sampling project or 0 if the criteria were not met; that is, whether the sampling project data could 

provide the Agency with the stated information. The total score is computed as the weighted sum of the 

criteria scores, or:

TotalScore = ∑ wj

18

j=1

× Scorej,

where Wj is the criteria weight and Scorej is the criteria score. In other words, for each sampling project, 

the responses (1 or 0) for each question are multiplied by the weight for that question, and then added 

together to get the overall weighted score for the sampling project. The different sampling projects 

could then be ranked according to those weighted scores, which represent the relative potential 

benefits of the different sampling projects. Detailed findings on the potential benefits of ongoing 

sampling projects, including raw data from SME evaluation of all existing sampling projects, are included 

in Appendix D: Detailed Findings on the Potential Benefits of Ongoing Sampling Projects.

Table 1 presents the results of the scoring, ranking and weighting of each sampling project in order of its 

weighted score, from highest to lowest. The effect of the weighting is shown in Table 1, where the rank 

according to the weighted score for a given project is higher than the unweighted ranking (e.g., NRP 43, 

Order 43). This effect is more pronounced in Figure 2, which shows the unweighted and weighted scores 

ROC Weight (Initial) ROC Weight (Avg Ties)

1.a.i 1.0 0.180 0.180

4.a 3.0 0.129 0.121

2.a 3.0 0.112 0.121

1.a.iii 3.5 0.095 0.095

4.c 5.0 0.081 0.076

3.a 5.0 0.071 0.076

4.d 6.0 0.061 0.061

1.a.ii 7.5 0.052 0.049

1.b.i 7.5 0.045 0.049

4.b 9.0 0.039 0.036

1.a.iv 9.0 0.033 0.036

3.b 11.0 0.027 0.027

2.b 11.5 0.023 0.021

3.c 11.5 0.018 0.021

1.a.v 12.0 0.014 0.014 1. Assess Individual Establishments ...

1.b.ii 13.0 0.010 0.010 2. Conduct Investigations

4.f 17.0 0.006 0.006 3. Modernize Agency Policies...

4.e 17.5 0.003 0.003 4. Track Agency Progress ...

Rank Order Centroid Criteria Rank

I I 

J J I 

J I I 

J I I 

J J ] 

J J ] 

] ] 

I ] ] 

I ] ] 

J ] I 

J J I 

J 

I - ,_ 

I 

I 

I 
20

for each sampling project from highest to lowest weighted score. There are several sampling projects 

that have four criteria identified as being applicable, but the weighted score varies—and the black line 

diverges—depending on which criteria are scored with a one. For instance, a score of 1 was given to 

both the NARMS sampling projects and the State residue testing projects for four questions (shown as 

multicolored bars in Figure 2, ranges are Sampling Program 129-136 and 79-87 respectively). However, 

when the weighted value for each individual question was applied (shown as a black line in Figure 2), the 

overall weighted score for NARMS decreased, while the overall weighted score for the State residue 

sampling projects increased. A more drastic example of how the weighting affected final ranking is 

demonstrated by the final weighted score for the pathology sampling projects, which decreased from 

fulfilling four criteria, to having an overall score of less than 1. These examples highlight the benefits of 

utilizing a weighted scoring approach versus an unweighted approach. 

Table 1: Summary of Sampling Projects Ranked by Unweighted and Weighted Score

Unweighted Weighted Unweighted Weighted Unweighted Weighted Unweighted Weighted

1 MT43 8 0.60 4 1 EM36 5 0.40 8 1 NRP_ST_S 4 0.40 121 F_TU_COM01 3 0.20

2 MT60 8 0.60 4 2 EM37 5 0.40 8 2 NRP_SW_S 4 0.40 122 FAMR01 2 0.18

3 MT64 7 0.57 4 3 KIS 5 0.48 8 3 NRP_YC_S 4 0.40 123 AMR01 2 0.16

4 MT65 7 0.57 4 4 NRP_BC 5 0.48 8 4 NRP_YT_S 4 0.40 124 EXP_LV_ABX 2 0.16

5 RTEPROD_RAND 7 0.57 4 5 NRP_BS 5 0.48 8 5 IMPABNCONT 3 0.31 125 EXP_LV_HORM 2 0.16

6 RTEPROD_RISK 7 0.57 4 6 NRP_BV 5 0.48 8 6 IMPAMRBEEF 3 0.31 126 EXP_LV_SOY 2 0.16

7 INTCONT_LM_E 7 0.53 4 7 NRP_DC 5 0.48 8 7 IMPFISH_CH_E 3 0.31 127 FOODCHEM 2 0.16

8 INTCONT_LM_M 7 0.53 4 8 NRP_FFV 5 0.48 8 8 IMPFISH_CH_W 3 0.31 128 UNKSUB 2 0.16

9 INTCONT_LM_W 7 0.53 4 9 NRP_GO 5 0.48 8 9 F_CH_CARC01 4 0.32 129 NARMS_BC 4 0.26

1 0 INTCONT_SA_E 7 0.53 5 0 NRP_HC 5 0.48 9 0 F_TU_CARC01 4 0.32 130 NARMS_DC 4 0.26

1 1 INTCONT_SA_M 7 0.53 5 1 NRP_HF 5 0.48 9 1 FLISTERIA 4 0.29 131 NARMS_HF 4 0.26

1 2 INTPROD_LM_E 7 0.53 5 2 NRP_MS 5 0.48 9 2 RLMENVC_EL 4 0.31 132 NARMS_MS 4 0.26

1 3 INTPROD_LM_M 7 0.53 5 3 NRP_NFFV 5 0.48 9 3 RLMENVC_MWL 4 0.31 133 NARMS_ST 4 0.26

1 4 INTPROD_LM_W 7 0.53 5 4 NRP_OBT 5 0.48 9 4 RLMENVC_WL 4 0.31 134 NARMS_SW 4 0.26

1 5 INTPROD_SA_E 7 0.53 5 5 NRP_RS 5 0.48 9 5 RLMENVR_EL 4 0.31 135 NARMS_YC 4 0.26

1 6 INTPROD_SA_M 7 0.53 5 6 NRP_SH 5 0.48 9 6 RLMENVR_MWL 4 0.31 136 NARMS_YT 4 0.26

1 7 MT44T 7 0.53 5 7 NRP_ST 5 0.48 9 7 RLMENVR_WL 4 0.31 137 IMPSPECIESID 3 0.16

1 8 MT05 6 0.51 5 8 NRP_SW 5 0.48 9 8 RES_FI 5 0.37 138 LO_CH_CARC01 2 0.15

1 9 HC_CH_CARC01 8 0.52 5 9 NRP_YC 5 0.48 9 9 AMS_PROD_RTE 3 0.32 139 LO_CH_COM01 2 0.15

2 0 HC_CH_COM01 8 0.52 6 0 NRP_YT 5 0.48 100 EXP_CH_MSK01 4 0.23 140 LO_CH_MSK01 2 0.15

2 1 HC_CPT_LBW01 8 0.52 6 1 FRTESALMONEL 4 0.39 101 EXP_CPT_OT01 4 0.23 141 LO_CPT_LBW01 2 0.15

2 2 HC_TU_CARC01 8 0.52 6 2 EGGIMP 4 0.39 102 EXP_CPT_QH01 4 0.23 142 LO_CPT_OT01 2 0.15

2 3 HC_TU_COM01 8 0.52 6 3 IMVRTE 4 0.39 103 EXP_PK_COM02 4 0.23 143 LO_CPT_QH01 2 0.15

2 4 COMPLIAN 7 0.46 6 4 MT08 4 0.39 104 EXP_PK_ICT02 4 0.23 144 LO_TU_CARC01 2 0.15

2 5 RLMCONT_EL 5 0.46 6 5 MT51 4 0.39 105 EXP_PK_NCT02 4 0.23 145 LO_TU_COM01 2 0.15

2 6 RLMCONT_MWL 5 0.46 6 6 INTENV_LM_E 6 0.38 106 EXP_TU_MSK01 4 0.23 146 RE_CH_CARC01 2 0.15

2 7 RLMCONT_WL 5 0.46 6 7 INTENV_LM_M 6 0.38 107 IMP_PORK 3 0.23 147 LO_TU_MSK01 2 0.15

2 8 RLMPRODC_EL 5 0.46 6 8 INTENV_LM_W 6 0.38 108 IMP_POULTRY 3 0.23 148 FDS05 2 0.13

2 9 RLMPRODC_MWL 5 0.46 6 9 INTENV_SA_E 6 0.38 109 IMPFISH_MI 3 0.23 149 IMPPATH 4 0.18

3 0 RLMPRODC_WL 5 0.46 7 0 INTENV_SA_M 6 0.38 110 IMPHORMONES 2 0.19 150 PATH_LIVESTK 4 0.18

3 1 MT06 6 0.41 7 1 CG_RES_EL 4 0.40 111 IMPMETALS 2 0.19 151 PATH_OTHER 4 0.18

3 2 MT44 6 0.41 7 2 CG_RES_MWL 4 0.40 112 IMPPESTICIDE 2 0.19 152 PATH_POULTRY 4 0.18

3 3 MT52 6 0.41 7 3 CG_RES_WL 4 0.40 113 IMPRESEGG 2 0.19 153 PATH_PRODUCT 4 0.18

3 4 MT53 6 0.41 7 4 CG_SHOW_MWL 4 0.40 114 IMPRESFR_EL 2 0.19 154 ABNCONT 1 0.02

3 5 NRP_FS 6 0.50 7 5 CG_SHOW_WL 4 0.40 115 IMPRESFR_WL 2 0.19 155 FDS01 1 0.01

3 6 EM31 5 0.40 7 6 NRP_BC_S 4 0.40 116 IMPRESPR_EL 2 0.19 156 EXP_LV_NUTR 1 0.01

3 7 EM32 5 0.40 7 7 NRP_BV_S 4 0.40 117 IMPRESPR_ML 2 0.19 157 SPECID 1 0.01

3 8 EM33 5 0.40 7 8 NRP_DC_S 4 0.40 118 EXP_FI_MIC01 4 0.21

3 9 EM34 5 0.40 7 9 NRP_HF_S 4 0.40 119 F_CPT_LBW01 3 0.20

4 0 EM35 5 0.40 8 0 NRP_MS_S 4 0.40 120 F_CU_COM01 3 0.20

Potential Benefit Score Potential Benefit Score

Order Project Name Order Project Name

Potential Benefit Score

Order Project Name

Potential Benefit Score

Order Project Name

c=J D D □ c=J D D □ CJ CJ D □ CJ CJ D □ CJ CJ □ □ CJ CJ □ □ CJ CJ □ □ CJ CJ □ □ CJ CJ □ □ CJ CJ □ □ CJ CJ □ □ CJ CJ □ □ CJ CJ □ □ CJ CJ □ □ CJ CJ □ □ CJ CJ □ □ CJ CJ □ □ D CJ D □ D CJ □ □ D CJ □ □ D □ □ D D □ □ D D □ □ CJ D □ □ CJ D □ □ CJ D □ □ CJ D □ □ CJ D □ D 

CJ D □ □ CJ D □ □ D D □ □ D D □ □ D D □ □ D D □ I 

D D □ I 

D D □ I 

D D □ I 

D D □ D D □ D D □ 
21

Figure 2. Unweighted and Weighted Scores

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0

2

4

6

8

10

12

14

16

18

1591 31 72 12 52 93 33 74 14 54 95 35 76 16 56 97 37 78 18 58 99 39 7 101 105 109 113 117 121 125 129 133 137 141 145 149 153 157 Weighted Score Number of Criteria

Sampling Program

Total Potential Benefit Score 

4 f

4 e

4 d

4 c

4 b

4 a

3 c

3 b

3 a

2 b

2 a

1b ii

1bi

1av

1aiv

1aiii

1aii

1ai

I 

nin111111111111111111 mmmmrr 

-

-

-

-

mlllllli 

--

-

-

-

7//h 

-

-

-
22

By design, and consistent with FSIS’ mission, sampling projects related to adulterant testing ranked the 

highest in this analysis. Specifically, the for-cause sampling projects for Lm in RTE products and food 

contact surfaces were ranked the highest, followed by the follow-up and risk-based E. coli O157:H7 and 

other adulterant Shiga Toxin E. coli (STEC) sampling projects for raw ground beef. The Agency’s 

Pathogen Reduction/Hazard Analysis and Critical Control Point (PR/HACCP) verification sampling 

projects (for Salmonella and Campylobacter in poultry carcasses) were ranked just below the projects 

that address adulterants, indicating that the weighting of the sampling projects successfully aligned with 

the Agency’s purpose and mission.

The lowest scoring sampling projects were the only projects with an affirmative response to any of the 

following questions: test for food defense-related threat agents (Question 1av), provide inspection 

program personnel (IPP) with information on animal diseases that are diseases for which a carcass 

would be condemned (Question 1bi), determine the hazard or risk of an emerging pathogen, chemical,

or other agent in an FSIS-regulated product (Question 3b), provide IPP information on animal diseases 

for which FSIS conducts surveillance from an animal health perspective (Question 4f).

In general, the sampling projects with the highest scores are those that directly align with the Agency’s 

current priorities (identifying adulterants, reducing pathogen positive rates). FSIS’ exploratory 

microbiological and chemical residue sampling projects generally received mid-level rankings, reflecting 

the importance the Agency puts on those projects. 

As can be seen in Table 1 and Figure 2, the ranked, weighted scores are well spread among the sampling 

projects, with weighted scores ranging from 0.01 to 0.68. About 38% of the sampling projects scored 

below 0.3, indicating those provide limited information to the Agency for decisions. It is important to 

note, however, that this ranking only considers one phase of the evaluation of a sampling project— the 

potential benefits. Other aspects are examined in the other phases of this evaluation—Phase 3,

whether the sampling project is actually providing those benefits, Phase 4, whether the data are being 

used, and Phase 5, what the associated costs are for running sampling projects. These other aspects also 

need to be considered when evaluating sampling programs. In addition, there are instances in which 

other external considerations, such as international trade, other ramifications, and practicality should be 

taken into account when evaluating sampling projects, and those considerations are not included in this

ranking of potential benefits. Therefore, neither the raw nor the weighted score should be deterministic 

for sampling programs; that is, a sampling program should not be eliminated solely because it has a low 

weighted score. 

Recommendations

• FSIS should reexamine projects with a weighted-benefit ranking below 0.3 to determine whether 

improvements could be made to strengthen the value of those projects or whether the project 

should be discontinued. The reexamination should take into account the information obtained in 

other phases of this evaluation, as well as other external considerations.

• Moving forward, the criteria and weighting for the potential benefits (Box 1) should be included in 

requests for new sampling projects or modifications to existing projects. 
23

Phase 3: Assess Whether Each Sampling Project, as Implemented, Can 

Provide the Intended Information and Benefits

Approach and Methodology

After evaluating the potential benefits of each sampling project through the qualitative criteria scoring in Phase 2, the 

SASR workgroup assessed whether the identified benefits are likely to be realized from implementation of the project 

(Phase 3). This phase is important because, regardless of the potential benefits, if a sampling project is not designed and 

implemented correctly, those benefits will not be realized. This Phase 3 analysis only applies to routine, algorithm-based 

sampling projects like Raw Ground Beef, Young Chicken Carcasses, and Ready-to-Eat (RTE) products. It does not apply to 

collector generated, follow-up sampling or retail projects; such sampling programs are not statistically designed and, 

therefore, cannot be evaluated on a statistical basis. Importantly, that means they cannot be used to estimate national 

trends, prevalence or other such metrics.

The workgroup, in conjunction with Agency statisticians and analysts, developed a series of questions to help determine 

whether a sampling project’s design, or, in the case of sampling projects undergoing Agency consideration, planned 

design, is adequate to provide the information sought. The SASR workgroup used several central tenets in reviewing the 

design of sampling projects. First, the number of samples allocated for each project should be as small as possible while 

still providing the necessary analytical information. Although collecting more samples produces more accurate estimates 

of pathogen rates and of establishment-specific and industry-wide performance, there can be diminishing returns to 

collecting large numbers of samples. FSIS must balance the need for accurate estimates with effective use of funds, IPP 

inspection activities, and cost to industry (from destructive sampling). 

There are two major factors that drive the required number of samples for FSIS sampling projects: 1) policy goals and 2) 

the statistical validity to draw conclusions related to those policy goals. Examples of statistical concerns are producing 

national estimates (e.g., prevalence) and monitoring progress towards Agency goals (e.g., Salmonella pathogen 

reduction in Chicken Carcasses). Examples of policy concerns include categorizing establishments (e.g., Salmonella in 

Chicken Carcasses) and specific requirements based on a public health concern.

Box 2 contains a summary of common statistical terms used throughout the text below for ease of understanding. 
24

Box 2: Overview of Statistical Terms

FSIS Pathogen Rate Estimation

FSIS publicly posts pathogen rate estimates for multiple sampling projects2,3. There are three different estimates 

produced: percent positive4, volume-weighted percent positive, and prevalence. On the webpage, FSIS defines the three 

estimates as follows:

• Percent positive (PP): The percentage of samples of a specific FSIS-regulated product 

with a specific pathogen. These calculations may vary significantly from the actual 

prevalence of the population, but FSIS cannot make a more precise calculation with the 

data currently available.

• Volume-weighted percent positive (VWPP): The VWPP for a specific product-pathogen 

pair is calculated by combining the production volumes (production volume can be 

defined as the amount (weight or numbers of head) of product produced per year) for 

establishments with their sampling results. This provides a more accurate estimation 

than a percent positive, but the data do not meet the statistical requirements to make a 

prevalence estimate. VWPP is more public health focused than PP, in that weighting 

results by volume reflects exposure of the public to specific pathogens.

 

2 https://www.fsis.usda.gov/wps/portal/fsis/topics/data-collection-and-reports/microbiology/sampling-project-results

3 Use of FSIS Regulatory Verification Sampling to Generate Prevalence Estimates, April 2012 https://www.fsis.usda.gov/wps/wcm/connect/56b2ccbdad57-4311-b6df-289822d28115/Prevalence_Estimates_Report.pdf?MOD=AJPERES

4

FSIS uses a logit transformation when producing 95% confidence intervals for pathogen rates. This transformation removes the possibility of a 

confidence interval overlapping 0% or 100%, by making the confidence interval “unbalanced.” For ease of discussion, this document does not 

utilize this transformation.

Confidence Interval: A range of values for a particular estimated parameter (i.e. pathogen rate), 

for which, at some level of confidence, the real value of the parameter is included. FSIS generally 

uses 95% confidence intervals for estimated pathogen rates (prevalence, or percent positive). 

Example Statement: The 95% confidence interval for the prevalence of Salmonella in Beef 

Manufacturing Trim was 1.1% to 3.78%.

Margin of error: A measure of an estimate’s precision. As the MOE decreases, number of samples 

required must increase.

Significance level (α): The significance level is the probability of deeming the new project 

successful at reducing pathogen levels when in fact it is not successful.

• Subtracting the significance level from one (1-α) yields the confidence level of the interval.

• For a 95% confidence interval, set the significance level to 0.05. 

• This is a Type I error in statistical contexts. 

• Smaller values of α require larger number of samples. 

Power: The power of a test is its ability to detect a change in prevalence that indeed has occurred

because of the new project.

o A common value for power is 0.80, or 80%.

o A larger number of samples provides more power.
25

• Prevalence: The estimated proportion, nationally, of a specific FSIS-regulated product 

with a specific pathogen.

FSIS definitions of these estimates vary slightly among publications at different points in time; for example, there are 

small differences between the current definitions and those referenced in the 2015 methodology paper.[1]

Footnote 2 from the publicly posted spreadsheets is applied to Processed Egg and Ready-to-Eat estimates, which are 

listed as volume weighted percent positive, “/2/. These calculations are made using the same calculation as prevalence. 

FSIS is not labeling these calculations prevalence because the data may not meet the statistical criteria for prevalence. If 

FSIS determines that the data do meet the statistical criteria, the label will be updated.” The criteria that must be met to 

call an estimate a “prevalence” are outlined in the Prevalence estimate report. If an estimate does not meet the criteria 

for estimating prevalence, for transparency, the specific criteria that are lacking should be documented.

The FSIS definition of a prevalence estimate suggests there is a minimum number of samples required for labeling an 

FSIS estimate as “prevalence” versus “volume weighted percent positive” or for not calculating a prevalence. However, 

no explicit standard is set, or referenced in any of the documentation. 

A direct effect of increasing the number of samples is to reduce the margin of error for a given estimate. The number of 

samples required to have a margin of error less than a pre-determined acceptable limit depends on the pathogen rate 

for the specific product/pathogen pairing in a sampling project. Rather than set a strict sample size limit, the SASR 

workgroup decided to base the calculations for the number of samples required on reaching a designated standard for 

the margin of error—see Table 2.

It is important to remember, however, that FSIS sampling projects often serve not only to estimate national prevalence, 

but also to assess an individual establishment’s control of pathogens. Therefore, determinations about the number of 

samples to collect should be based not only on the number needed to estimate prevalence in the overall industry, but 

also on the number of samples needed to monitor individual establishments.

Table 2: Designated Standards for the Maximum Margin of Error Calculations by Pathogen Rate

Pathogen Rate Maximum Margin of Error

Greater than 50% +/- 5%

From 1% to 50% +/- 1%

< 1% +/- the expected Pathogen Rate

Statistical Determination of Sample Size 

FSIS worked closely with MITRE to develop an Excel-based calculator tool to determine the required number of FSIS inplant samples. The tool allows users to produce preliminary estimates of the number of samples required for three 

common situations: 

• A confidence interval for a single proportion (e.g., prevalence) to within a specified margin of error

• An upper bound on the probability of a positive test result (e.g., adulterant monitoring)

• The difference between two proportions (e.g., impact of an industry standard on prevalence of a non-adulterant 

pathogen) 

It should be stressed that this tool is only the first step in determining the number of samples needed to accomplish the 

goals of each sampling project. Other analyses should be incorporated into the evaluation, such as needs for 

 

[1] FSIS Estimation of Pathogen Rates in the Population Represented by the Samples Collected in Each Sampling Project for Fiscal Year 2015

www.fsis.usda.gov/wps/wcm/connect/56b2ccbd-ad57-4311-b6df-289822d28115/

Prevalence_Estimates_Report.pdf?MOD=AJPERES
26

stratification (e.g., by establishment) or other statistical measures; policy requirements; public health concerns; and

technical, resource-based, and other limitations that could influence the final number of samples.

The design of the preliminary calculation for the number of samples required is purposely simple. The calculation 

estimates the largest number of samples required for a given set of parameters. Use of some complex statistical designs 

(stratification) or variance calculations (jackknife method) will produce confidence intervals with smaller margins of 

error for similar numbers of samples. However, for preliminary estimates and general planning purposes, the tool will 

provide useful estimates for the number of samples required to meet FSIS goals.

Confidence interval estimates

Confidence interval estimates are useful for establishing a range of plausible values around an estimated prevalence 

value. Three input values are needed to use the tool to determine the required number of samples for a pathogen rate 

estimate:

1. An estimate of the true prevalence of a pathogen in FSIS in-plant sampling

a. This input may be based on current data, previous studies, or similar studies. 

2. Maximum Margin of Error Estimate (MOE)

a. If, for example, prevalence must be estimated to within +/-0.10, or 10%, then 0.10 is the maximum 

MOE. 

3. Significance Level

a. FSIS generally uses an Alpha of 0.05 for confidence intervals.

Table 3 provides sample size estimates for a given pathogen rate and desired margin of error. More samples are 

required to produce a smaller MOE. For example, if the pathogen rate is 40%, 1,476 samples are needed for a MOE of 

+/- 2.5%. This means that with 95% confidence the pathogen rate observed fell between 37.5-42.5%. Whereas, 9,220 

samples are required to reduce the MOE to +/- 1% (39% - 41%). 

Table 3: Tool to estimate the number of samples required for a given pathogen rate and a set maximum margin of error, 

assuming a 95% confidence interval.

Pathogen 

Rate

Number of Samples Required for Different Maximum Margins of Error 

(+/- % range), Alpha=0.05

10% 5% 2.5% 1% 0.5% 0.1% 0.05% 0.01%

40% 93 369 1,476 9,220 36,879 921,951 3,687,801 92,195,012

30% 81 323 1,291 8,068 32,269 806,707 3,226,826 80,670,636
27

20% 62 246 984 6,147 24,586 614,634 2,458,534 61,463,342

10% 35 139 554 3,458 13,830 345,732 1,382,926 34,573,130

5% 73 292 1,825 7,299 182,470 729,878 18,246,930

1% 61 381 1,522 38,031 152,122 3,803,045

0.50% 192 765 19,112 76,446 1,911,126

0.10% 154 3,838 15,351 383,762

0.01% 385 1,537 38,411

Upper bound estimates

Upper bound estimates provide comparatively less information than interval estimates, but they also require smaller 

numbers of samples. Upper bound estimates are based on the idea that for any given sample size, if no positive test 

results are observed, then pathogen rate does not exceed some upper bound with known confidence. Two input values 

are needed to use the tool to determine the required number of samples for an upper bound estimate:

1. The upper bound of the pathogen rate. 

a. If there are no positive test results, this is the value the pathogen rate does not exceed.

2. Significance Level 

a. The Upper Bound estimates do not have a standard Alpha value. This differs from the Alpha=.05 

standard for confidence intervals.

The following is an example of estimating the number of samples required for an upper bound estimate. Assume that 

FSIS will take no action as long as the overall prevalence of a non-adulterant pathogen remains at or below some 

acceptable level, say 0.25, or 25%. The number of samples FSIS must collect so that, if there are no positive test results 

for that pathogen, the agency can be 95% confident that the pathogen’s prevalence does not exceed 25% is 11. The 

‘price’ of the small number of samples is confidence that the prevalence of the non-adulterant pathogen is somewhere 

at or below 25% if no positive results are detected. If two of the samples were positive, the estimated pathogen rate 

would be 18.18%, with a margin of error larger than 18.18%. The benefits for using the upper bound estimates to 

determine the number of samples required are found when the pathogen rate is very low.

The tool allows the user to easily compare sampling requirements for interval estimates and upper bound estimates. 

Table 4: Upper bound number of samples required calculations by pathogen rate and confidence.

Pathogen 

Rate

Number of Samples Required for Different Alphas (confidence 

intervals)

0.1 0.05 0.025 0.01 0.005 0.001 0.0005 0.0001

3% 76 99 122 152 174 227 250 303

2% 114 149 183 228 263 342 377 456

1% 230 299 368 459 528 688 757 917

0.50% 460 598 736 919 1,058 1,379 1,517 1,838
28

0.25% 920 1,197 1,474 1,840 2,117 2,760 3,037 3,680

0.10% 2,302 2,995 3,688 4,603 5,296 6,905 7,598 9,206

0.05% 4,605 5,990 7,376 9,209 10,594 13,813 15,199 18,417

0.01% 23,025 29,956 36,887 46,050 52,981 69,075 76,006 92,099

Table 4 provides upper bound sample size estimates for a given pathogen rate by varying confidence intervals. As an 

example of table interpretation, if FSIS analyzed 4,603 samples without finding any positives for the given pathogen, the 

Agency could be 99% confident that the pathogen rate does not exceed 0.10%. Basing the sample size on the upper 

bound number of samples requirement does not preclude the Agency from estimating the pathogen rate and producing 

a confidence interval. The resulting confidence interval will be wider than if the number of samples were set to optimize 

the interval, but with far fewer samples collected. For example, if the rate for the non-adulterant pathogen were 

actually 0.01%, 38,411 (see Table 3) samples would be required to reduce the margin of error to +/- .01%, but the 95% 

confidence interval from 4,603 samples would have a margin of error of +/- .029%.

Difference between two proportions test

Instead of monitoring occurrence of pathogen in FSIS testing with an interval or upper bound estimate, some projects 

may be designed with the goal of lowering pathogen prevalence in FSIS in-plant testing from current levels to meet 

targets. In this case, it is important to be able to establish a plausible correlation between the new project and the new 

prevalence level by testing for the difference between two proportions.5 The goal of this test is to allow FSIS to state the 

pathogen rate was reduced by a desired percentage due to industry’s response to some action by FSIS.

The following four input values are needed to use the tool to determine the number of samples required to show the 

difference between two proportions:

1. An estimate of the true, initial prevalence of a pathogen in FSIS in-plant testing (starting proportion) 

o This input may be based on current data, previous studies, or similar studies. 

2. A second proportion that is an estimate of the true, new pathogen rate in FSIS in-plant testing

o The pathogen rate expected because of a new project or action. 

o The difference between the first and second proportion is known as the minimum clinically relevant 

difference. 

o The tool assumes that a project must lower the pathogen rate to the value of the second proportion to 

be considered successful. 

3. The significance level (α) of the test. 

4. The power (1-β) of the test.6

o If the new project is a success, it is important that we collect enough data to verify that. 

If a project is being designed to reduce pathogen positive rates (Rate), to detect the difference, FSIS would compare the

rates from two different time periods. Thus, the total number of samples required would be the number of samples 

needed to determine Rate 1 plus the number of samples needed to determine the Rate 2. For example, FSIS would need 

to collect 1,091 samples per rate (2,182 total) to be able to detect a change in pathogen rate from 25% to 20% with 80% 

power and 95% confidence. 

Analysis

 

5 Testing for the difference between two proportions is a more general case of assessing the impact of a new standard. Assessing the impact of a 

new standard is covered in the next section of this document. 

6 The quantity β is the false negative rate of the test. It is analogous to α, commonly referred to as significance, which is the false positive rate of 

the test. 
29

The workgroup conducted focus groups with key members of each sampling workgroup to walk through each sampling 

project and, using the number of samples estimator tool developed by MITRE, assess whether enough samples were 

collected to determine, with statistical confidence, that the stated purpose of the project was achieved. For example, for 

sampling projects that were designed to assess pathogen prevalence in FSIS in-plant testing, is FSIS collecting enough 

samples to produce a 95% confidence interval with an acceptable margin of error?

Results

The discussion of results is organized into domestic sampling and import sampling. Domestic sampling is conducted to 

verify HACCP in producing establishments, while imports sampling is conducted as one part of the international 

equivalence process.

Domestic Sampling

Estimation of the pathogen prevalence in FSIS in-plant testing is important for several sampling projects, such as the 

poultry verification projects for carcasses, parts, and comminuted products. For pathogen-product pairs that have 

existing performance standards, such as Salmonella in chicken carcasses where the standard seeks a 25% reduction in 

illness, the number of samples estimated by the difference between two proportions test provides sound statistical 

results for planning purposes. 

Most FSIS sampling projects do not require strict adherence to the maximum margin of error requirements for 

estimation of pathogen prevalence to be useful to the Agency. For projects that seek to estimate pathogen rate where 

the current pathogen rates are exceedingly low, such as the current pathogen rate of 0.01% for E. coli O157:H7 in raw

ground beef, estimating the pathogen rate with an ideal margin of error requires far more samples to be collected and 

analyzed than the Agency has the resources to perform. 

However, many sampling projects could utilize the Upper Bound estimate to guide sample size decisions. At the 

establishment level, no adulterants are permitted, and because of FSIS and industry efforts, the number of positive tests 

for adulterants is very low. It is conceivable that no positive results may be found over a given fiscal year. When this 

occurs, FSIS is limited in the statements that it can make regarding the data because no confidence interval can be 

calculated without at least 1 positive and 1 negative observation. However, if 4,603 samples were taken, FSIS could state 

with 99% confidence that the percent positive did not exceed 0.1%. If 9,209 samples were taken, FSIS could state with 

99% confidence that the percent positive did not exceed 0.05%. 

As discussed earlier, FSIS does not have a standard for the maximum margin of error of a confidence interval for an 

estimate of a pathogen rate, particularly for a prevalence estimate. For purposes of this analysis, the workgroup used 

the parameters identified in Table 5.

Table 5: Parameters for a standard maximum margin of error of a confidence interval for an estimate of a pathogen rate

Pathogen Rate Maximum Margin of Error

Greater than 50% +/- 5%

From 1% to 50% +/- 1%

< 1% +/- the expected Pathogen Rate

For ease of reference, a portion of Table 4, Upper Bound Sample Size Calculations is reproduced in Table 6.

Table 6: Subset of Table 4 (Upper bound sample size calculations by pathogen rate and confidence)
30

Pathogen 

Rate

Alpha

0.1 0.05 0.025 0.01 0.005

0.50% 460 598 736 919 1,058

0.25% 920 1,197 1,474 1,840 2,117

0.10% 2,302 2,995 3,688 4,603 5,296

0.05% 4,605 5,990 7,376 9,209 10,594

0.01% 23,025 29,956 36,887 46,050 52,981

The workgroup reviewed current sample size allocations for a select group of FSIS sampling projects and the associated 

pathogen rate estimates and performed some sample size calculations based on that data. The sample size calculations 

shown below are based on simple statistical concepts. As such, they are meant to be a starting point for determining 

appropriate sample sizes. There are many reasons the sample size may be more or less than the number suggested by 

the calculations (policy, advanced statistical techniques, etc.). Additional information is located in Appendix E: Required 

Number of Samples Estimates for Published Pathogen Rates.

Overview of Sample Size Estimations for Select Sampling Projects

• Risk-Based Sampling of Raw Ground Beef or Veal Products – E. coli O157:H7 & Salmonella (MT43)

o For FY 2019, FSIS allocated 11,500 samples for MT43.

o FSIS estimates the prevalence of E. coli O157:H7 in FSIS in-plant testing under MT43 to be around 0.01%.

▪ 38,411 samples would be required to produce a confidence interval with a margin of error of 0.01% and an 

estimate of 0.01%. 

▪ If all 11,500 samples were negative in FY 2019, FSIS could state with 99.5% confidence the E. coli O157:H7 

rate did not exceed 0.05%.

o FSIS estimates the prevalence of Salmonella in FSIS in-plant testing under MT43 to be around 3.89%.

▪ 1,437 samples would be required to produce a confidence interval with a margin of error of 1% and an 

estimate of 3.89%.

▪ Two sets of 5,484 samples would be required (for example: 5,484 samples in FY18 and 5,484 samples in 

FY19) to be able to show a 25% reduction in Salmonella from 3.89% to 2.92% with 95% confidence and 80% 

power.

• Sampling of Bench Trim for further use in ANY raw, non-intact beef products (MT65) 

o For FY 2019, FSIS allocated 1,500 samples for MT65.

o FSIS estimates the prevalence of E. coli O157:H7 in FSIS in-plant testing under MT65 to be almost 0%. FSIS did 

not find a positive sample in calendar year 2018.

▪ At least one positive sample is required to produce a confidence interval.

▪ 38,411 samples would be required to produce a confidence interval with a margin of error of 0.01% and an 

estimate of 0.01%.

▪ If all 1,500 samples were negative in FY 2019, FSIS could state with 97.5% confidence the E. coli O157:H7 

rate did not exceed 0.25%.

o FSIS estimates the prevalence of Salmonella in FSIS in-plant testing under MT65 to be around a 0.66%.

▪ 696 samples would be required to produce a confidence interval with a margin of error of 0.66% and an 

estimate of 0.66%.

▪ Two sets of 35,356 samples would be required (for example: 35,356 samples in FY18 and 35,356 samples in 

FY19) to be able to show a 25% reduction in Salmonella from 0.66% to 0.5% with 95% confidence and 80% 

power.
31

• Sampling of Processed Egg Products (EM31-EM37) 

o For FY 2019, FSIS allocated 1,600 samples for all EM31-EM37.

o FSIS estimates the volume weighted percent positive of Lm in EM31-EM37 products to be almost 0%. FSIS did 

not find a positive sample in calendar year 2018.

▪ At least one positive sample is required to produce a confidence interval.

▪ 38,411 samples would be required to produce a confidence interval with a margin of error of 0.01% and an 

estimate of 0.01%.

▪ If all 1,600 samples were negative in FY 2019, FSIS could state with 97.5% confidence that the Lm rate did 

not exceed 0.25%.

o FSIS estimates the prevalence of Salmonella in EM31-EM37 products to be around a 0.01%.

▪ 38,411 samples would be required to produce a confidence interval with a margin of error of 0.01% and an 

estimate of 0.01%.

▪ If all 1,600 samples were negative in FY 2019, FSIS could state with 97.5% confidence that the Salmonella

rate did not exceed 0.25%.

• HACCP Verification for Young Chicken Carcasses (HC_CH_CARC01)

o For FY 2019, FSIS allocated 9,000 samples for HC_CH_CARC01.

o FSIS estimates the prevalence of Salmonella in young chicken carcasses to be around a 4.25%.

▪ 1,564 samples would be required to produce a confidence interval with a margin of error of 1% and an 

estimate of 4.25%.

▪ 2,735 samples required for both sets of data to be able to show a 25% reduction (the stated Healthy People 

2020 goal) in Salmonella from 7.5% to 5.63% with 95% confidence and 80% power.

o FSIS estimates the prevalence of Campylobacter in young chicken carcasses to be around a 29.5%.

▪ 7,990 samples would be required to produce a confidence interval with a margin of error of 1% and an 

estimate of 29.5%.

▪ 549 samples would be required for both sets of data to be able to show a 25% reduction in Campylobacter

from 29.5% to 22.13% with 95% confidence and 80% power.

• Sampling for Ground and Other Comminuted Turkey (not Mech. Separated) (HC_TU_COM01)

o For FY 2019, FSIS allocated 1,500 samples for HC_TU_COM01.

o FSIS estimates the prevalence of Salmonella in comminuted turkey products to be around a 25.41%.

▪ 7,281 samples would be required to produce a confidence interval with a margin of error of 1% and an 

estimate of 25.41%.

▪ 631 samples would be required for both sets of data to be able to show a 25% reduction (the stated Healthy 

People 2020 goal) in Salmonella from 19.9% to 14% with 95% confidence and 80% power.

o FSIS estimates the prevalence of Campylobacter in comminuted turkey products to be around a 2.71%.

▪ 1,013 samples would be required to produce a confidence interval with a margin of error of 1% and an 

estimate of 2.71%.

▪ 7,914 samples would be required for both sets of data to be able to show a 25% reduction in Campylobacter

from 2.71% to 2.03% with 95% confidence and 80% power.

The number of samples required for almost all of FSIS’ domestic sampling projects can be justified by the underlying 

statistical requirements. There are instances, particularly for the adulterant testing projects, where there may be an 

opportunity to reduce sampling allocations, but the policy requirements of those projects and the implications of 

reducing sampling must be considered before committing to any reduction.

It should be noted the National Residue Program (NRP) already utilizes the upper bound sample size calculations for 

determining the number of samples required for the various NRP sampling projects.
32

Import Sampling

All imported shipments of meat, poultry, and egg products that enter the United States are presented to FSIS for reinspection. Point-of-entry re-inspection, including sampling, is one element of a three-part approach FSIS uses to 

evaluate and verify the on-going equivalence of an exporting country’s food safety inspection system. FSIS checks every 

imported shipment for eligibility, certification, transportation damage, proper labeling, carton count, and general 

condition. 

FSIS import re-inspection sampling uses PHIS to allocate types of inspections by country, process category, product 

category, product group and species. PHIS provides the ability to increase or decrease re-inspection of products by 

country or establishment using three levels of inspection: Normal, Increased, and Intensified. For normal sampling, FSIS 

collects and analyzes samples for pathogens, species identification, and drug and chemical residues at set intervals. 

Import re-inspection is performance-based in that failed types of inspections trigger additional like inspections on 

subsequent lots received from those foreign establishments – known as intensified level of re-inspection. The decision 

to increase the level of inspection above normal is an agency/management decision.

While the import sampling plan is modeled after the domestic sampling project, there are major differences in 

implementation. Domestic establishments operate and produce products eligible for sampling at predictable intervals 

and quantities. As such, it is possible to assign sampling tasks in a routine fashion and have confidence in the number of 

samples collected. Import sampling is dependent on establishments in foreign countries exporting product to the United 

States. In addition, domestic sampling is conducted to evaluate PR/HACCP in FSIS-regulated establishments. 

Import sampling is only one segment of a three-part process to continually evaluate and verify the equivalency of an 

exporting country’s food safety inspection system. It is supplemental to the sampling programs that the foreign country 

conducts; to be deemed equivalent, the foreign country’s sampling program would already have been determined to 

have been equivalent. Import sampling is not designed to evaluate the equivalence of the foreign country. For a specific 

country product pair, there may be only a handful of lots shipped each year. Therefore, it is not possible to collect 

enough samples of a particular product for most importing countries to make any sound statistical determinations about 

any pathogen or residue.

Imported lots are refused entry if the shipment fails to meet U.S. requirements. For shipments refused entry for reasons 

significant to Public Health, such as an adulterant being detected upon re-inspection, FSIS: 

1. Notifies the foreign central competent authority (CCA) and requests an investigation, a root cause analysis, and 

an assessment of corrective actions as verified by the foreign CCA; 

2. Evaluates the response as part of assessing the foreign country’s on-going equivalence; and

3. Uses the information to plan future audits performed as part of on-going equivalence verification. 

For shipments that fail a lab sampling type of inspection, samples of all subsequent lots of like product and country and 

foreign establishment are collected until a specified number of negative results are received. For adulterants, the chance 

of a positive is likely to be so low that the intensification in sampling rate will act as a deterrence. The minimum number 

of intensified samples following a pathogen positive is 15. If all were negative, then FSIS could conclude, with 95% 

confidence, the pathogen rate for the product/country/processing establishment during intensified sampling did not 

exceed 19%. 

For products that yield non-regulatory results, such as a Salmonella positive in raw chicken parts, FSIS does not reject 

the product. FSIS does not make any statistical determinations regarding the product, the performance of the foreign 

establishment, or the country’s equivalence based on import sampling. To obtain a statistically based sample, 

significantly more import samples would need to be collected, the PHIS import module would require a rework and, 

most likely, a significant amount of work with international stakeholders would be necessary.
33

Recommendations

General Recommendations

• Pathogen workgroups should evaluate new and ongoing sampling projects to optimize sample allocation. To meet 

this overarching recommendation, the workgroups should take the following steps:

o Using the tool as a starting point, the workgroups should determine the appropriate number of samples to 

analyze to ensure that any statistical benefits (e.g. maximum margin of error, upper bound claims, or 

comparison of rates) can be realized.

o The workgroups should also determine the minimum number of samples required to meet policy goals 

(e.g. number of samples needed for establishment categorization).

o The number of samples required to realize the statistical and policy benefits should be determined 

annually, be documented and included in the workgroup submission to the FSIS Annual Sampling Plan. 

• The workgroup developed a series of questions that can be used to determine whether a sampling project’s design 

is adequate to realize the potential benefits for which the sampling project is being initiated or conducted. Those 

questions are presented in Box 2. The workgroup recommends these questions be incorporated into the Sampling 

Plan Change Request Form. 

• FSIS should consider the statistical claims that can be made based on the number of samples collected and analyzed

when determining the required number of samples for each sampling project.

• FSIS should more clearly document the reasoning for the required number of samples and the potential statistical 

claims that are associated with each sampling project. 

• FSIS should develop standards for determining whether a prevalence estimate based on FSIS in-plant sampling can 

be calculated for each sampling project, including identifying whether there is a maximum margin of error for calling 

an estimate a “prevalence.” 

• For all projects, FSIS should clearly document the reasons a prevalence estimate based on FSIS in-plant sampling can 

or cannot be calculated.

Sampling Project Specific Recommendations

• The egg product workgroup has evaluated the sampling model and has suggested updated methodology. The 

Agency has not made a final decision. FSIS should complete review of the proposal and implement any changes by 

FY 2020.

• Import sampling, as currently conducted by FSIS, cannot be used to make any statistical claims. To obtain a 

statistically based sample, significantly more samples would need to be taken, the PHIS import module would 

require a rework, and, most likely a significant amount of work with international stakeholders would be necessary. 

FSIS should evaluate import sampling to maximize benefit to the Agency.

Other Recommendations

• FSIS sampling programs use different methods for determining production volume categories of establishments. FSIS 

should develop a standard method for determining production volume categories for establishments.
34

Box 3: Questions to Help Consider whether Sampling Project Designs are Adequate

What background is available to help inform the design?

• How will the Agency use the data? That is, what is the intent of the sampling? (Please use list of potential 

benefits discussed in Box 1.)

• Is there any evidence that the pathogen or chemical is in the product or is this an exploratory sampling 

project?

• If there is evidence, is it known at what frequency the analyte is present?

• What evidence is there that there is an overall risk?

• Are there existing data to help inform the sampling design?

• Has there been a risk assessment or other analysis conducted to support the design?

• What is the severity of the outcome?

• How concerned do we need to be if there are positives?

• Can you ‘piggy-back’ on another sampling project? 

What are the characteristics of the samples being collected and the analysis?

• Is the sample a rinsate, grab sample, or some other type of sample?

• What is the volume of product that will be evaluated and how was that determined?

• What does that volume of product represent?

• What is the level of detection at that volume?

• What is the sensitivity and specificity of the test method?

Are you trying to assess individual establishments or countries?

• Do you need to differentiate between different classes/sizes of establishments?

• Do you need to differentiate between different countries?

• Do you need to evaluate individual establishments?

• Is there a minimum number of samples you want from each establishment?

• Is there a maximum number of samples that you can collect from each establishment?

Are you trying to look at national rate or trends?

• Are you looking to determine the baseline level of a hazard/compound in a specific product? If so, why?

• What percent positive do you want to be able to detect with confidence, or at what percent positive are 

you concerned? (Note: this gets to what percent positive we are trying to estimate, which informs the 

number of samples for a given level of confidence.)

• How confident do you need to be in your conclusions from this sampling project? General trend? Detect a 

50% change? Detect a 10% change? 

• Are you looking for a threshold above which you think something is likely to occur or is there a minimal 

analyte level below which you don’t care?

What number of samples (i.e., what sampling size) needs to be collected for the desired purpose?

• What is the best estimate of the national pathogen rate?

• Are you looking to produce a national pathogen rate estimate?

YES

o What is the maximum margin of error acceptable for the estimate?

o Are you looking to compare multiple time periods, if so, what amount of change are you looking 

to measure?

NO

o What is the highest pathogen rate you are willing to accept the pathogen rate does not exceed if 

there are no positive sampling results?

o What level of certainty do you want that the pathogen rate does not exceed that level?

• If there is a targeted number of samples for each establishment, how many samples are required to reach 

those targets?
35

Phase 4: For On-Going Sampling Projects, Determine Whether FSIS Has 

Used the Results for Agency Decisions

Approach and Methodology

In Phase 4, the SASR workgroup conducted a survey of SMEs, as well as the pathogen and chemical residue workgroups,

to identify what decisions, or in some cases, types of decisions the Agency has made based on the results of the 

sampling projects. The purpose of this compilation was to better elucidate the ways in which FSIS has used the data it 

has collected. All sampling projects were developed with a purpose in mind, but whether they achieved that purpose 

and have been used to help shape Agency decision-making required further evaluation. 

Examples of decisions the Agency makes using results from sampling projects include responding to residue violations 

with enforcement actions, using data on microbial trends to drive policy decisions, and using import sample results in 

future evaluations of a country’s equivalence.

Analysis

Included in this compilation are the Project Code, Project Name, Description of How Data are Used in Agency Decisions, 

the number of samples planned in the FSIS FY 2018 Annual Sampling Plan, the number of samples analyzed in the FY

2018 Annual Plan, and the number of samples planned for the FSIS FY 2019 Annual Sampling Plan.

Results

The results of this compilation are presented in Appendix E: Required Number of Samples Estimates for Published 

Pathogen Rates

FSIS posts pathogen rates for select sampling projects quarterly. Reported estimates are either prevalence, volumeweighted percent positive, or straight percent positive. This appendix uses values from CY2018 published values because 

it has the most current pathogen rates to demonstrate the number of samples required for confidence interval 

estimates and to determine a difference between two proportions. 

The number of samples required to meet the maximum margin of error for the estimated pathogen rate is based off the 

requirement discussed in Phase 3 on Table 5: Parameters for a standard maximum margin of error of a confidence 

interval for an estimate of a pathogen rate . 

The number of required samples for these projects tends to be much higher than the number of samples allocated in the 

Annual Sampling Plan. In a few instances, the number of samples required is much lower. The projects where the 

required number of samples is considerably lower than the Annual Sampling Plan are highlighted.

The number of samples required for each proportion is calculated from either the current estimated pathogen rate, or 

from the actual FSIS documentation. For current estimated rates, the goal listed is a 25% pathogen (the stated Healthy 

People 2020 goal) rate reduction. For projects with specific goals, the number of samples required is based on the target 

pathogen rate, not the current estimated pathogen rate.
36

Table E1: Estimated Number of Samples Required for Projects with Published Pathogen Rates

Workgroup Project Pathogen

FY 2019

Annual 

Sampling 

Plan

Interval Estimates Difference Between Tw

Estimated 

Pathogen 

Rate CY 2018

Max 

Margin 

of Error

Significance 

Level 

(alpha)

Required 

# of 

Samples 

Initial 

Pathogen 

Rate

Target 

Pathogen 

Rate

SCCG HC_CH_CARC01 Salmonella 9,000 4.49%1 1% 0.05 1,791 7.50% 5.63% 

Campylobacter 13.03%3 1% 0.05 4,354

HC_CH_LBW01 Salmonella 9,000 12.99%1 1% 0.05 4,342 28% 18% 

Campylobacter 15.40%3 1% 0.05 5,005

HC_CH_COM01 Salmonella 2,500 38.32%1 1% 0.05 9,080 49% 34% 

Campylobacter 7.96%3 1% 0.05 2,815

HC_TU_CARC01 Salmonella 2,000 0.53%1 0.53% 0.05 721 1.70% 1.275% 

Campylobacter 0.51%3 0.51% 0.05 750

HC_TU_COM01 Salmonella 1,500 23.21%1 1% 0.05 6,847 19.90% 14% 

Campylobacter 2.53%3 1% 0.05 948 1.20% 1.0% 

EXP_CH_MSK01 Salmonella 150 80.95%3 5% 0.05 237 80.95% 60.71% 

Campylobacter 62.50%3 5% 0.05 361 62.50% 46.88% 

EXP_TU_MSK01 Salmonella 150 48.48%3 1% 0.05 9,595 48.48% 36.36% 

Campylobacter 9.09%3 1% 0.05 3,175 9.09% 6.82% 

EXP_CPT_QH01 Salmonella 120 15.60%3 1% 0.05 5,058 15.60% 11.70% 

Campylobacter 20%3 1% 0.05 6,147 20% 15% 

EXP_CPT_OT01 Salmonella 360 54.15%3 5% 0.05 382 54.15% 40.62% 

STEC MT43 E. coli O157:H7 11,500 0.01%1 0.01% 0.05 38,411 

Salmonella 3.36%1 1.00% 0.05 1,248 3.36% 2.52% 

MT60 E. coli O157:H7 3,750 0.15%1 0.15% 0.05 2,558

non-O157 STEC 0.25%1 0.25% 0.05 1,533

Salmonella 1.86%1 1.00% 0.05 702 1.86% 1.39% 

MT64 E. coli O157:H7 1,050 0.17%3 0.24% 0.05 2,256

Salmonella 7.11%3 1.00% 0.05 2,538 7.11% 5.33% 

MT65 E. coli O157:H7 1,500 0.00%3--% 0.05 --

Salmonella 0.58%3 0.58% 0.05 659 0.58% 0.44% 

Pork EXP_PK_ICT02 Salmonella 1,521 10.57%3 1.00% 0.05 3,632 10.57% 7.93% 

EXP_PK_NCT02 Salmonella 1,272 7.15%3 1.00% 0.05 2,551 7.15% 5.37% 

EXP_PK_COM02 Salmonella 1,704 20.88%3 1.00% 0.05 6,347 20.88% 15.66% 

Egg 

Products 

EM Salmonella 1,600 0.00%2--% 0.05 --

L. monocytogenes 0.00%2--% 0.05 --

RTE RTEPROD_RAND Salmonella 7,400 0.01%2 0.01% 0.05 38,411

L. monocytogenes 0.10%2 0.10% 0.05 3,838

RTEPROD_RISK Salmonella 7,400 0.02%2 0.02% 0.05 19,204

L. monocytogenes 0.01%2 0.01% 0.05 38,411
37

1 – Prevalence 2 – Volume Weighted Percent Positive 3 – Percent Positive 4 – Number of samples required to detect the 

difference officially stated by FSIS

Appendix F: Summary, by sampling project, of analyses conducted using 

results from each project. In general, results from most of FSIS’ current sampling projects are 

used to make regulatory decisions, such as enforcement actions, or to inform Agency policy decisions. 

There are many ways in which FSIS’ sampling projects achieve their intended goals, and information 

from those projects often trigger immediate actions by the Agency. For example, the identification of a 

product positive for an adulterant (E. coli O157:H7 or non-O157 STEC identified in a ground beef sample

(MT43)) triggers a District Office notification through the FSIS Biological Information Transfer and E-Mail 

System (BITES). The result is put into the FSIS System Tracking E. coli O157:H7 – Positive Suppliers 

(STEPS), and the Agency decides whether a product retention, detention, or recall is necessary.7 

Similarly, for an establishment subject to the Salmonella pathogen reduction performance standards, 

results from a Salmonella sampling (e.g., in chicken carcasses (HC_CH_CARC01)) are used to determine 

whether or not an establishment has met the performance standard, and these results are also subject 

to a monthly public posting of the establishment’s category status.8 

However, this evaluation identified sampling projects from which results have not been used to inform 

FSIS policy. For example, for many of the existing exploratory sampling projects—such as the Agency’s 

 

7 https://www.fsis.usda.gov/wps/wcm/connect/18fd8926-62dd-42c0-aa1c8109e87f4170/Ecoli_FSIS_Actions.pdf?MOD=AJPERES

8 https://www.fsis.usda.gov/wps/wcm/connect/ebf83112-4c3b-4650-8396-

24cc8d38bf6c/10250.1.pdf?MOD=AJPERES
38

exploratory sampling for chicken parts (quarter and half carcasses; EXP_CPT_QH01*)—at the time of 

this evaluation, no formal analyses have been conducted with the data produced from the project and 

results have not been used to inform FSIS policies.

Project Level Evaluations and Improvements

Concurrently with this analysis, the Salmonella and Campylobacter Coordination Group (SCCG)

undertook an evaluation of the very low volume, religious exempt, and mechanically separated poultry 

exploratory sampling projects. The evaluation was conducted with the spirit of the SASR 

recommendations and ultimately resulted in FSIS deciding to end the very-low volume and religious 

exempt projects. A decision for the mechanically separated poultry projects is pending the results of an 

establishment survey regarding product usage.

Recommendations

• The SCC should revise the current sampling request SCC form to include the following updates:

• Include a schedule for routine evaluation (with a formal analysis plan) of each sampling project 

to determine whether the results of the project are being used to inform FSIS policy.

• Include a question to prompt the requestor to evaluate whether the necessary information 

could be obtained by updating an existing sampling project.

• Review existing exploratory/baseline sampling and make determinations to either discontinue 

sampling or incorporate the exploratory portion of the sampling into a routine sampling project. 

• For projects that will be developed after the other recommendations are in place, ensure that there 

is a sampling timeline that includes a specified sunset date for the project.
39

Phase 5: Assessment of Costs Across Sampling Projects

Approach and Methodology

In Phase 5, the SASR workgroup estimated the cost of each sampling project to compare the relative 

costs among the different sampling projects. The cost estimates include both an estimate of the FSIS 

field personnel costs for collecting the sample at an establishment or in-commerce, and the FSIS 

laboratory costs associated with analyzing the sample.

The workgroup consulted with OFO personnel to gather information on the cost of inspection personnel 

time for collecting samples. It was determined that the absolute time that it takes to collect and ship a 

sample varies greatly, both between the different types of samples that are collected, as well as from 

establishment to establishment, because of the different physical sizes and layouts of establishments. It 

should be noted that inspection personnel collect samples during their workday assignment in lieu of 

performing some other inspection activity identified in PHIS as having a lower priority.

Therefore, the workgroup categorized the time for different types of sample collection as long, medium,

or short to differentiate the costs of collection of samples. The costs of shipping the samples was 

approximately the same, regardless of the type of sample and was not used in the relative cost 

estimates. FSIS OPHS laboratory personnel provided estimates of the cost of analyzing samples for 

different product–pathogen pairs. 

The workgroup estimated the relative costs of the different sampling projects by estimating the cost per 

sample and multiplying that cost by the number of samples collected for that sampling project. The 

following types of costs were considered when estimating the cost per sample:

I. The cost of inspection personnel time for collecting the samples for the different projects,

II. Estimates of the costs of laboratory personnel time for conducting the analyses, and 

III. Where possible, estimates of the costs of the equipment and reagents for each sample. 

Analysis

Costs Associated with FSIS Field Personnel Collecting Samples

The time FSIS IPP spend collecting samples varies depending on a variety of issues, including the product 

or type of sample being collected, the sample collection method, the size of the establishment, and the 

data entry requirements in PHIS. For the purposes of this analysis the SASR workgroup used OFO’s

methodology for its workforce planning, as described below. 

OFO aggregates its work measurement data into four groupings: 

• Direct inspection time (i.e., actual observation or hand-on task time), 

• Indirect inspection time (e.g., data entry, research, and analytical time), 

• Internal travel time (i.e. inside the plant), and 

• External travel time (i.e. outside the plant). 

FSIS allocates 30 minutes direct time for conducting sampling tasks, except the N-60 sampling task, 

which is allotted 60 minutes. The N-60 sample collection method is used with the MT60 and MT65 

project codes. The indirect time factored into the work assignment calculation by applying an indirect 

multiplier, therefore the task indirect time varied between establishments and work assignments.
40

This analysis does not include internal or external travel time.

Table 7: Indirect Inspection Time Estimates for Large Establishments

Work Measurement 

Grouping

All Sample Collection 

Methods*

N 60 Method

Direct Inspection Time 30 minutes 60 minutes

Indirect Inspection Time 54 minutes 108 minutes

Internal Travel Time Not Included Not Included

External Travel Time Not Included Not Included

*Except N60

Finally, to estimate the cost of collecting samples, the workgroup multiplied the time estimates by the 

base salary of a GS-10 step 1 full time employee with no locality pay, and included the standard 

multiplier from the OMB Circular A-769for estimating benefits and administrative overhead of 36.25%. 

The FSIS employee hourly rate is an approximation because samples can be collected by FSIS employees 

ranging from grades GS-9 through GS-12 at any of the 10 steps within the grades and at locations 

throughout the country with various locality pay rates.

Costs Associated with FSIS Laboratories Analyzing Samples 

A second aspect of the costs associated with FSIS sampling projects is the laboratory costs of analyzing 

the different types of samples. FSIS laboratory personnel provided estimates of the costs of different 

types of analyses performed by the FSIS labs. Cost estimates for a sampling project overall, however, 

were complicated by many factors, not the least of which was that total costs vary based on the number 

of positive samples that need further characterization, for example whole genome sequencing (WGS). 

This cost estimate does not include the laboratory labor costs for conducting the analysis, and only 

includes the direct costs for sample screening and confirmatory testing. At the time of the evaluation

there was not enough information regarding labor hours to include in the report. OPHS field laboratories 

do not have a robust cost tracking system to account for all laboratory related costs, such as costs 

associated with instrumentation. 

Since positives can greatly increase the cost of a sampling project by requiring further confirmatory 

testing, the costs for confirmatory testing of positives and further characterization are added to the 

costs of screening the negatives, to obtain an overall cost for a sampling project. It should also be noted 

that the confirmatory testing costs vary by analyte, and the rates of positives can vary year-to-year and 

by project. 

To determine an average cost for a microbiological sample, FSIS summed the direct costs for analyzing 

samples, such as collection supplies, shipping and storage costs, and screening tests. Then multiplied

these summed costs by the total number of samples planned for the year. The average cost also 

includes additional costs for confirmatory testing or further characterization. The expected number of 

samples for these costs are derived from the estimated percent positive rate for that project and are 

estimated based on historical data. 

 

9 https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/circulars/A76/a076.pdf
41

The direct testing costs and the confirmatory testing are summed and then divided by the total number 

of samples analyzed to obtain an overall average for each sampling project. This analysis was completed 

for all projects designed to test for E. coli O157:H7, other STEC, Salmonella, Campylobacter, and Lm. The 

NARMS costs were also included in the estimate for a microbiological sample. The current microbiology 

OPHS cost models did not include equipment costs as they were primarily focused on consumables. 

These costs were then summed across all projects and divided by the total number of samples analyzed 

to get a total average cost for a microbiological sample. 

A similar methodology was used for estimating an overall average cost for chemical or drug residue 

testing for a single sample. Since the methods for these types of analyses are quite different than 

microbiological testing, they are estimated separately. Chemical and drug residue testing can also be 

conducted using a variety of methods and FSIS tests for a variety of compounds. Again, the costs were 

averaged across several different methods and compounds, including the Multi-Residue Method (MRM), 

pesticide method, and tests for sodium, speciation, fat content, and others. Reagent costs per sample 

were multiplied by the number of samples planned for each method. Costs for confirmatory testing 

were added based on the reagent costs per confirmatory sample and the historical percent positive 

rates. Costs for equipment maintenance contracts were also added. These costs were then divided by 

the total number of samples tested to obtain an average direct cost for chemical or drug residue testing.

Similarly, this cost estimate does not include the laboratory labor costs for conducting the analysis, and 

only includes the direct costs for sample screening and confirmatory testing. 

As with both the microbiology and chemistry cost estimates, a similar approach was used to create the 

pathology cost estimates. The difference between pathology and the other estimates is that FSIS only 

performs two types of pathology analysis, advanced meat recovery (AMR) and normal diagnostic 

analysis. These two cost estimates include the consumables necessary for analysis, as well as the cost to 

send supplies to the field to gather the samples. Labor costs are not included. Unlike the previous two 

estimates, the pathology costs were not averaged, since there are designated projects that can 

distinguish the specific type of analysis to be performed. 

Results

A complete list by project of the cost assessment is provided in Appendix G: Details of Costs Associated 

with Sampling Projects. Table 8 summarizes the five-different analysis/collection combinations that 

were found.

Table 8: Summary of Cost by Laboratory Analysis and Sample Collection

Laboratory Analysis Type Laboratory Costs 

per sample ($)

Sample 

Collection

Minutes

Field Personnel

Hourly Rate ($)

Field Personnel 

Costs per sample 

($)

Total Cost 

Estimate per 

sample ($)

Microbiological 82.17 54

31.53

28.38 110.55

Microbiological (N60) 82.17 108 56.75 138.92

Chemical Residue 115.06 54 28.38 143.44

Pathology (Diagnostic) 30.38 54 28.38 58.76

Pathology (AMR) 96.41 54 28.38 124.75

Recommendations

• FSIS should utilize OPHS resource subject matter experts to create and continuously refine a 

repository to track resource requirements by lab and project.
42

• New sampling project sponsors and the SCC should ensure that:

• a sampling project’s design has been optimized, 

• how the information from the sampling project will be used by the Agency is 

considered,

• and consider the overall costs associated with the sampling plan before the decision to 

institute the sampling project is made.
43

Appendix A: Evaluation Participants

Participation included a broad cross-section of the agency, including the Office of the Chief Financial 

Officer (OCFO), the Office of Planning, Analysis, and Risk Management (OPARM), the Office of Field 

Operations (OFO), the Office of Investigation, Enforcement, and Audit (OIEA), the Office of Policy and 

Program Development (OPPD), the Office of Public Health Science (OPHS), and the Office of 

International Coordination (OIC). 

The workgroup reached out to subject matter experts (SMEs) to help describe and assess individual 

sampling projects, to provide cost estimates, and to help select appropriate statistics for evaluating 

whether a sampling project is designed in a way to meet the potential benefits. Participants 

predominantly acted through one or more of the working groups, or as SMEs on various topics, as 

identified below:

Evaluation Team Leads:

OPHS: Rebecca Fields, Joanna Zablotsky Kufel

OPARM: Jackson Crockett

OCFO: Matthew Gonzales

Evaluation Workgroup Members

OPHS OPARM

Emilio Esteban Jackson Crockett (Co-lead)

David Goldman (former FSIS) Dee Zamora

Rebecca Fields (Co-lead) Paul Dolan

Neal Golden JenAlyse Arena

Joanna Zablotsky Kufel (Co-lead) Terri Nintemann 

Mary Katherine Crews

Denise Eblen OIEA

Vincent Fayne

Don Anderson (retired)

OCFO

Sarah Lynch (former FSIS)

Matthew Gonzales (Co-lead) OA

Babar Bilal Todd Reed

Greg Albers Michelle Catlin (Co-lead)

Mike Toner

OPPD

OFO Roberta Wagner

Phil Bronstein Rachel Edelstein

Jessica Pulz John Linville

Lisa Volk Jennifer Webb

Amber Pasko Catherine Rockwell

Michelle Kemper

Consultants Stephanie Defibaugh-Chavez

Phil Derfler Erika StappKamotani

MITRE Corp. Meryl Silverman
44

Appendix B: Description of Ongoing Sampling Projects

Typically, when FSIS initiates a sampling project, the Agency states its purpose and description in the 

Federal Register notice announcing the project. Table B1 in this appendix provides those descriptions 

and cites the FSIS documents which contain either that description or the regulation that applies.

 

Table B1. Description of Ongoing Sampling Projects

Project Code Sampling Project Name

Purpose and Description of Sampling Project

(from Federal Register Notice Unless Otherwise Indicated)

ABNCONT Abnormal Containera Samples are collected ad hoc when inspection program personnel (IPP) observes an abnormal 

container being used (Thermal Processing).b

AMR01 Advanced Meat Recovery Product Beef AMR product is defined as product containing central nervous tissue (CNS) or CNS-type 

tissues. AMR product is defined as not “beef;” it cannot be used as an ingredient of a “meat 

food product" (9 CFR 318.24). The AMR01 project is restricted to product produced from beef 

skull or vertebral column bones because these are the most likely products to contain CNS 

tissues or CNS-type tissue (Directive 7160.3). Note that FSIS has determined it is not necessary 

to establish a verification sampling project for SRM removal (72 FR 38711).

AMS_PROD_RTE AMS RTE Canada EV Project - Product 

Collected by FSIS

In 2015, FSIS and AMS Export Verification (EV) Project designed this collaborative testing project 

to verify establishments' control of pathogens in closed-faced sandwiches produced under

voluntary FSIS inspection that are intended for export to Canada (80 FR 67382).

CG_RES_EL; 

CG_RES_MWL; 

CG_RES_WL;

Collector Generated - Residue -

Eastern Lab/ Midwestern Lab/ 

Western Lab

Established in 1967 (77 FR 39896), this annual sampling project was developed by FSIS, FDA, 

EPA, and ARS, AMS, and CDC based on investigations, veterinary drug inventories, and on-farm 

visits. The group creates a list of chemical compounds for testing and ranks them by public 

health risk and regulatory concern. Then the group considers FSIS lab capacity and analytical 

methods (77 FR 39895) to verify establishments’ control of animal drug residues, pesticides, 

environmental contaminants, and any other chemical hazards in and on meat and poultry 

products. The National Residue Projects (NRP) also provides national data on chemical residue 

testing results to support risk assessment, enforcement, and educational activities (Directive 

10,800.1). 

CG_SHOW_MWL; 

CG_SHOW_WL

Collector Generated-Residues-Show 

Animals-Midwest Lab/Western Lab

Established in 1967 (77 FR 39896), this annual sampling project was developed by FSIS, FDA, 

EPA, and ARS, AMS, and CDC based on investigations, veterinary drug inventories, and on-farm 

visits. The group creates a list of chemical compounds for testing and ranks them by public 

health risk and regulatory concern. Then the group considers FSIS lab capacity and analytical 

methods (77 FR 39895) to verify establishments’ control of animal drug residues, pesticides, 

environmental contaminants, and any other chemical hazards in and on meat and poultry 

products. The National Residue Projects (NRP) also provides national data on chemical residue 

testing results to support risk assessment, enforcement, and educational activities (Directive 

10,800.1).

COMPLIAN Investigative Sampling Samples are collected for products in commerce that are suspected to be adulterated or 

misbranded (FSIS Directive 8010.1).

EGGIMP Import-Egg Products-Salmonella and 

Lm

Sampling for imports of pasteurized liquid, frozen egg products, and dried egg products 

(Directive 9900.2).

EM31 Egg Product Sampling-PasteurizedEgg Whites-Salmonella and LmThe Egg Monitoring (EM) group of product codes tests egg products that must be pasteurized 

before they leave the official plant (21 U.S.C. §1034 and §1036). 

EM32 Egg Product Sampling-PasteurizedWhole Egg or Yolks - Salmonella and 

Lm

EM33 Egg Product Sampling-PasteurizedWhole Eggs with Added Yolks or 

Whole Egg Blends-Salmonella and Lm

EM34 Egg Product Sampling-PasteurizedWhole Eggs or Yolks with > 2% salt or 

sugar added-Salmonella and Lm

EM35 Egg Product Sampling-PasteurizedDried Yellow Egg Products -

Salmonella and Lm

EM36 Egg Product Sampling-PasteurizedDried Egg Whites - Salmonella and 

Lm

EM37 Egg Product Sampling-PasteurizedPan Dried Egg Whites - Salmonella 

and Lm
45

Project Code Sampling Project Name

Purpose and Description of Sampling Project

(from Federal Register Notice Unless Otherwise Indicated)

EXP_CH_MSK01 Exploratory Sampling for 

Mechanically Separated Chicken

Exploratory sampling project whose goal is to determine the prevalence of Salmonella in 

comminuted poultry to develop performance standards for these products (77 FR 72686).

EXP_CPT_OT01 Exploratory Sampling for Chicken 

Parts - Other Parts

Sampling of other raw chicken parts to gain additional information about the prevalence and 

the microbiological characteristics of Salmonella and Campylobacter in those products (80 FR 

3940). 

EXP_CPT_QH01 Exploratory Sampling for Chicken 

Parts - Quarter and Half Carcasses

Sampling of quarter and half raw chicken carcasses to gain additional information on the 

prevalence and the microbiological characteristics of Salmonella and Campylobacter in those 

products (80 FR 3940).

EXP_FI_MIC01 Domestic Siluriformes Sampling for 

Microbiology

In 2015, FSIS announced (80 FR 79231) it would conduct sampling and testing of Siluriformes

fish and fish products for Salmonella (76 FR 10439). 

EXP_LV_ABX Label Verification – Antibiotic Free

Label verification for antibiotic free claims for products eligible for sampling under the raw 

chicken parts project codes, HC_CPT_LBW01 and HC_CPT_QH01 (75 FR 82148). Label 

verification as described in FSIS Notice 26-18.

EXP_LV_HORM Label Verification – Hormone Free

Label verification for hormone free claims eligible for raw ground beef products that are eligible 

for Escherichia coli O157:H7 testing (75 FR 82148). Label verification as described in FSIS Notice 

26-18.

EXP_LV_NUTR Label Verification for Nutrient 

Content - Raw Ground Beef 

Label verification for nutrient content in raw ground beef samples that are eligible for E. coli 

O157:H7 testing (75 FR 82148). Label verification as described in FSIS Notice 26-18.

EXP_LV_SOY Label Verification – Allergens Sampling of ready-to-eat (RTE) products that are eligible for sampling under the 

RTEPROD_RAND project code (Directive 10,240.4)

EXP_PK_COM02 Baseline Sampling for Pork -

Comminuted (Ground, Mechanically 

Separated, and Other Comminuted)

Exploratory sampling of raw pork products for pathogens of public health concern as well as for 

indicator organisms (80 FR 3940).

EXP_PK_ICT02 Baseline Sampling for Pork - Intact 

cuts

EXP_PK_NCT02 Baseline Sampling for Pork - NonIntact Cuts

EXP_TU_MSK01 Exploratory Sampling for

Mechanically Separated Turkey

Sampling to determine the prevalence of Salmonella in comminuted poultry and to develop 

performance standards for these products (77 FR 72686).

F_CH_CARC01

Follow-up sampling of Chicken 

Carcasses

Follow-up testing, where applicable, of chicken carcass samples after positives for Salmonella

and Campylobacter (80 FR 3948).

F_CH_COM01

Follow-up sampling of Comminuted 

Chicken

Follow-up testing, where applicable, of comminuted chicken products after positives for 

Salmonella and Campylobacter (81 FR 7285)

F_CPT_LBW01

Follow-up sampling of Chicken Parts 

– Legs, Breasts and Wings

Follow-up testing, where applicable, of chicken part samples – legs, breasts, and wings, after 

positives for Salmonella and Campylobacter (81 FR 7285)

F_TU_CARC01

Follow-up sampling of Turkey 

Carcasses

Follow-up testing, where applicable, of turkey carcass samples after positives for Salmonella

and Campylobacter (80 FR 3948).

F_TU_COM01

Follow-up sampling of Comminuted 

Turkey

Follow-up testing, where applicable, of comminuted turkey products after positives for 

Salmonella and Campylobacter (81 FR 7285)

FAMR01 Follow-up Sampling to Advanced 

Meat Recovery Positive

FAMR01 is follow-up testing of positive AMR01 samples.

FDS01 Food Defense Sampling - Meals 

Ready-to-Eat

Directive 5420.1 

FDS05 Food Defense Sampling- Inauguration 

Testing

Directive 5420.1

FLISTERIA Import-Follow-up to Listeria 

Monocytogenes Positive

Follow-up sampling for the FSIS risk-based verification testing project to assess the 

effectiveness of RTE operations in controlling L. monocytogenes (69 FR 34221) of imported RTE 

products after positives for L. monocytogenes.

FOODCHEM Collector-Generated Food Chemistry 

Samples

Collector-generated samples for food chemistry analyses (Directive 10,630.1)

FRTESALMONEL Import-Follow-up to Salmonella 

Positive in RTE Product

Follow up testing of imported ready-to-eat (RTE) meat, poultry, and egg products after positives 

for Salmonella. 

HC_CH_CARC01/ 

HC_TU_CARC01

HACCP Verification for Young Chicken 

Carcasses/Turkey Carcasses

HC_CH_CARC01/ HC_TU_CARC01 is verification sampling used to confirm performance 

standards for Salmonella and Campylobacter in young chicken and turkey carcasses (75 FR 

27288; Note: 9 CFR 381.94(b) amended 79 FR 49566), as well as to estimate national 

prevalence.
46

Project Code Sampling Project Name

Purpose and Description of Sampling Project

(from Federal Register Notice Unless Otherwise Indicated)

HC_CH_COM01/ 

HC_TU_COM01

Sampling for Ground and Other 

Comminuted Chicken/Turkey (not 

Mechanically Separated)

Verification sampling used to confirm performance standards for Salmonella and 

Campylobacter in ground and other communicated chicken and turkey products (not 

Mechanically separated) as well as to estimate national prevalence (80 FR 3944).

HC_CPT_LBW01 Sampling for Chicken Parts – Legs, 

Breasts, and Wings

Sampling of leg, breast and wing chicken parts to gain additional information on the prevalence 

and the microbiological characteristics of Salmonella and Campylobacter in those products (80 

FR 3940).

IMP_PORK Imported Raw Pork Product Sampling Sampling raw intact and non-intact pork products for Salmonella species (FSIS Notice 93-16).

IMP_POULTRY Import Sampling of Raw and NRTE 

Poultry Products

Sampling of imported poultry carcasses, imported raw chicken parts, and imported NRTE 

comminuted chicken and turkey for Salmonella and Campylobacter (80 FR 3940).

IMPABNCONT Import-Abnormal Container Samples are collected ad hoc when inspection program personnel (IPP) observes an abnormal 

container being used for an imported product (Thermal Processing).b

IMPAMRBEEF Import - AMR Product - Beef Random sampling of meat products as they enter the United States as part of the equivalency 

determination for foreign countries, for BSE/SRMs (72 FR 38719).

IMPFISH_CH_E; 

IMPFISH_CH_W

Import Siluriformes Chemistry 

Sampling - Eastern 

Laboratory/Western Laboratory

Residue sampling of imported Siluriformes fish, like FSIS does for imported meat products, to 

ensure the safety of imported catfish products (80 FR 79231). 

IMPFISH_MI Import Siluriformes Microbiology 

Sampling

(Residue sampling of imported Siluriformes fish, like FSIS does for imported meat products, to 

ensure the safety of imported catfish products (80 FR 79231). Sampling for Siluriformes 

currently only includes analysis of Salmonella under this project.

IMPMETALS National Residue Plan - Import 

Sampling - Metals - Eastern Lab

Established 1967 (77 FR 39896), the annual sampling plan is developed by FSIS, FDA, EPA, and 

ARS, AMS, and CDC based on investigations, veterinary drug inventories, and on-farm visits. The 

group creates a list of chemical compounds for testing and ranks them by public health risk and 

regulatory concern. Then the group considers FSIS lab capacity and analytical methods (77 FR 

39895) to verify establishments control animal drug residues, pesticides, environmental 

contaminants, and any other chemical hazards in and on meat and poultry products. The NRP 

also provides national data on chemical residue testing results to support risk assessment, 

enforcement, and educational activities (Directive 10,800.1).

IMPPESTICIDE National Residue Plan - Import 

Sampling - Pesticides - Western Lab

Established in 1967 (77 FR 39896), the annual sampling project is developed by FSIS, FDA, EPA, 

and ARS, AMS, and CDC based on investigations, veterinary drug inventories, and on-farm visits. 

The group creates a list of chemical compounds for testing and ranks them public health risk 

and regulatory concern. The group then considers FSIS lab capacity and analytical methods (77 

FR 39895) to verify establishments control animal drug residues, pesticides, environmental 

contaminants, and any other chemical hazards in and on meat and poultry products. The 

National Residue Program (NRP) also provides national data on chemical residue testing results 

to support risk assessment, enforcement, and educational activities (Directive 10,800.1).

IMPRESEGG Import Residue Sampling - Egg 

Products

IMPRESFR_EL; 

IMPRESFR_WL

National Residue Plan - Import 

Sampling - Fresh Product

IMPRESPR_EL; 

IMPRESPR_MWL

National Residue Plan - Import 

Sampling - Processed Products

IMPSPECIESID Import-Species Identification Verification of the species claim on imported products. (Directive 9900.6)

IMVRTE Import-Micro Pathogen Sampling of 

RTE Products

IMVRTE is a risk-based verification testing project to assess the effectiveness of RTE operations 

in controlling L. monocytogenes (2004; 69 FR 34221). Generally, FSIS expects to collect, for L. 

monocytogenes testing, just one sample unit of RTE product from a production lot at an 

establishment selected for sampling.

INTCONT_LM_E; 

INTCONT_LM_M; 

INTCONT_LM_W

For-Cause Sampling- RTE Food 

Contact Surface-Listeria

This is for-cause sampling on RTE product and environmental samples. Risk-based verification 

testing project assesses the effectiveness of RTE operations in controlling L. monocytogenes

(2004; 69 FR 34221). FSIS expects to collect food contact surface samples and environmental 

samples mainly from operations that have a history of problems associated with the proper 

control for L. monocytogenes or Salmonella depending on the project.

INTCONT_SA_E; 

INTCONT_SA_M

For-Cause Sampling- RTE Food 

Contact Surface-Salmonella

INTENV_LM_E; 

INTENV_LM_M; 

INTENV_LM_W

For-Cause Sampling- Non-Food 

Surface-Listeria

INTENV_SA_E;

INTENV_SA_M

For-Cause Sampling- Non-Food 

Surface-Salmonella

INTPROD_LM_E; 

INTPROD_LM_M; 

INTPROD_LM_W

For-Cause Sampling- RTE ProductListeria

INTPROD_SA_E; 

INTPROD_SA_M

For-Cause Sampling-RTE ProductSalmonella

KIS KIS - Samples from In-plant Testing KIS (Kidney Inhibition Swab) sampling is intended for inspector-generated residue testing. FSIS 

laboratories use multi-residue screening methods on tissue samples submitted from positive 

KIS tests (Directive 10800.1)
47

Project Code Sampling Project Name

Purpose and Description of Sampling Project

(from Federal Register Notice Unless Otherwise Indicated)

LO_CH_CARC01 Very Low Volume Sampling for 

Chicken Carcasses

FSIS collects samples from eligible products 3–4 times per year from eligible establishments (80 

FR 3946). Establishments are eligible for this project if they have been exempted from 

LO_CH_COM01 Very Low Volume Sampling for Salmonella verification testing if they produce less than 1,000 pounds per day of product.

Ground and Other Comminuted 

Chicken (not Mechanically Separated)

LO_CH_MSK01 Very Low Volume Sampling for 

Mechanically Separated Chicken

LO_CPT_LBW01 Very Low Volume Sampling for Raw 

Chicken Parts - Legs, Breast, Wings

LO_CPT_OT01 Very Low Volume Sampling for Raw 

Chicken Parts - Other Parts

LO_CPT_QH01 Very Low Volume Sampling for 

Chicken Parts - Quarters/Halves

LO_TU_CARC01 Very Low Volume Sampling for 

Turkey Carcasses

LO_TU_COM01 Very Low Volume Sampling for 

Ground and Other Comminuted 

Turkey (not Mechanically Separated)

LO_TU_MSK01 Very Low Volume Sampling for 

Mechanically Separated Turkey

MT05 Raw Ground Beef or Veal Sampling -

Retail

FSIS currently samples and tests various raw ground beef products (including veal products) for 

E. coli O157:H7 (1999; 64 FR 2804). The project sampling is done at inspected establishments 

and retail stores (2014; 79 FR 32436) and are co-analyzed for Salmonella.

MT06 Raw Ground Beef or Veal Sampling -

Retail - Follow-up

Based on suspected cause of E. coli O157:H7 contamination and the establishment’s corrective 

action, FSIS determines the appropriate number of follow-up samples to collect and (2002; 67 

FR 62333). (For Salmonella, see MT05 above)

MT08 Import-Sampling of Raw Ground or 

Comminuted Beef or Veal Product

FSIS currently samples and tests various raw ground beef products (including veal products) for 

E. coli O157:H7 (1999; 64 FR 2804). The project sampling is done at inspected establishments 

and retail stores. (For Salmonella, see MT05 above)

MT43 Risk-based Sampling of Raw Ground 

Beef or Veal Products - E. coli 

O157:H7 & Salmonella

FSIS currently samples and tests various raw ground beef products (including veal products) for 

E. coli O157:H7 (1999; 64 FR 2804). The project sampling is done at inspected establishments 

and retail stores. (For Salmonella, see MT05 above)

MT44 Follow-up Risk-based Sampling of 

Positive Raw Ground Beef or Veal 

Sample -E. coli O157:H7/Salmonella

Based on suspected cause of E. coli O157:H7 contamination and the establishment’s corrective 

action, FSIS will determine the appropriate number of follow-up samples to collect and test 

(2002; 67 FR 62333). (For Salmonella, see MT05 above)

MT44T Follow-up E. coli Sampling of Raw 

Ground Beef, Trimmings or 

Components (Traceback)

FSIS uses this sampling project to conduct verification activities at establishments that supply 

intact product to grinding establishments when the Agency determines that a supplier may be 

responsible for E. coli O157:H7-positive ground product (67 FR 62332). FSIS determines the 

number of follow-up samples to collect and test based on the suspected cause of E. coli

O157:H7 contamination and the establishment’s corrective action. 

FSIS conducts verification tests on trim when the Agency finds ground product at a grinder that 

receives product from outside sources positive for E. coli O157:H7 and can identify the supplier. 

(For Salmonella, see MT05 above)

MT51 Import-Raw Beef Manufactured 

Trimmings or Components for use in 

Ground Beef or Beef Products

FSIS uses this project to test trimmings, other source materials for imported non-intact product, 

and carcasses and parts that will be processed into non-intact product based on the observation 

that one of the best ways to control the risk of E. coli O157:H7 contamination is to test products 

while they are still intact (67 FR 62332). FSIS uses this project to test the effectiveness of these 

controls. (For Salmonella, see MT05 above)

MT52 Follow-up Sampling of Suppliers of 

Raw Ground Beef Trim or 

Components

FSIS uses this sampling project to conduct verification activities at establishments that supply 

intact product to grinding establishments when the Agency determines that a supplier may be 

responsible for E. coli O157:H7-positive ground product (2002, 67 FR 62332)

FSIS conducts verification tests on trim when the Agency finds ground product at a grinder that 

receives product from outside sources positive for E. coli O157:H7 and FSIS can identify the 

supplier. (For Salmonella, see MT05 above)
48

Project Code Sampling Project Name

Purpose and Description of Sampling Project

(from Federal Register Notice Unless Otherwise Indicated)

MT53 Follow-up Sampling of Beef Mfg. Trim 

or Other Raw Ground Beef or Beef 

Patty Components

FSIS uses this sampling project to conduct verification activities at establishments that supply 

intact product to grinding establishments when the Agency determines that a supplier may be 

responsible for E. coli O157:H7-positive ground product (67 FR 62332).

FSIS conducts verification tests on trim when the Agency finds ground product at a grinder that 

receives product from outside sources positive for E. coli O157:H7 and FSIS can identify the 

supplier. (For Salmonella, see MT05 above)

MT60 Sampling of Beef Manufacturing 

Trimmings

FSIS uses this project to testing trimmings, other source materials for non-intact product, and 

carcasses and parts that will be processed into non-intact product based on the observation 

that one of the best ways to control the risk of E. coli O157:H7 contamination is to test products 

while they are still intact. FSIS testing would be used to verify the effectiveness of these 

controls. (67 FR 62332)

FSIS also conducts routine verification testing for six non-O157 Shiga toxin-producing E. coli

(STEC), in addition to E. coli O157:H7, in raw beef manufacturing trimmings beginning June 4, 

2012. (For Salmonella, see MT05 above)

MT64 Sampling of Raw Ground Beef or Beef 

Patty Components (other than trim)

FSIS uses this project to test trimmings, other source materials for non-intact product, and 

carcasses and parts that will be processed into non-intact product based on the observation 

that one of the best ways to control the risk of E. coli O157:H7 contamination is to test products 

while they are still intact. FSIS testing would be used to verify the effectiveness of these 

controls. (For Salmonella, see MT05 above) (67 FR 62332)

MT65 Sampling of Bench Trim for further 

use in ANY raw, non-intact beef 

products

FSIS uses this project to test trimmings, other source materials for non-intact product, and 

carcasses and parts that will be processed into non-intact product based on the observation 

that one of the best ways to control the risk of E. coli O157:H7 contamination is to test products 

while they are still intact. FSIS testing would be used to verify the effectiveness of these 

controls. (For Salmonella, see MT05 above) (67 FR 62332)

NARMS_BC NARMS-Sampling-Beef Cows The NARMS sampling projects are part of a national public health surveillance system that 

tracks antimicrobial susceptibility among enteric bacteria from humans, retail meats, and food 

animals (Directive 10,100.1). The projects were established in 1996 as a partnership between 

the FDA, CDC, and FSIS to better understand the emergence, persistence, and spread of 

antimicrobial resistance among foodborne bacteria.

Samples are analyzed for Salmonella, Campylobacter, generic E. coli, and Enterococcus, and 

antimicrobial susceptibility testing is conducted on isolates obtained from these samples.

NARMS_DC NARMS- Sampling-Dairy Cows (Paired 

with RESNARMS_DC)

NARMS_HF NARMS- Sampling-Heifers

NARMS_MS NARMS- Sampling-Market Swine

NARMS_ST NARMS- Sampling-Steers

NARMS_SW NARMS- Sampling-Sows

NARMS_YC NARMS- Sampling-Young Chickens 

(Paired with RESNARMS_YC)

NARMS_YT NARMS- Sampling-Young Turkeys

NRP_BC National Residue Project Sampling -

Beef Cows

Established 1967 (77 FR 39896), the National Residue Sampling Projects are annual sampling 

projects developed by FSIS, FDA, EPA, and ARS, AMS, and CDC based on investigations, 

veterinary drug inventories, and on-farm visits. The group creates a list of chemical compounds 

for testing and ranks them public health risk and regulatory concern. The group then considers 

FSIS lab capacity and analytical methods (77 FR 39895) to verify establishments control animal 

drug residues, pesticides, environmental contaminants, and any other chemical hazards in and 

on meat and poultry products. The NRP also provides national data on chemical residue testing 

results to support risk assessment, enforcement, and educational activities (Directive 10,800.1).

NRP_BC_S National Residue Project - Beef 

Cows - State

NRP_BS National Residue Project Sampling -

Bull/Stag

NRP_BV National Residue Project Sampling -

Bob Veal

NRP_BV_S National Residue Project - Bob Veal -

State

NRP_DC National Residue Project Sampling -

Dairy Cows

NRP_DC_S National Residue Project - Dairy Cows 

- State

NRP_FFV National Residue Project Sampling -

Formula-Fed Veal

NRP_FS National Residue Project Sampling –

Feral Swine

NRP_GO National Residue Project Sampling -

Goats

NRP_HC National Residue Project Sampling -

Heavy Calf

NRP_HF National Residue Project Sampling -

Heifers

NRP_HF_S National Residue Project - Heifers -

State
49

Project Code Sampling Project Name

Purpose and Description of Sampling Project

(from Federal Register Notice Unless Otherwise Indicated)

NRP_MS National Residue Project Sampling -

Market Swine

NRP_MS_S National Residue Project - Market 

Swine - State

NRP_NFFV National Residue Project Sampling -

Non-Formula-Fed Veal

NRP_OBT National Residue Project - Old 

Breeder Turkeys

NRP_RS National Residue Sampling Project -

Roaster Swine

NRP_SH National Residue Project Sampling -

Sheep

NRP_ST National Residue Project Sampling -

Steers

NRP_ST_S National Residue Project - Steers -

State

NRP_SW National Residue Project Sampling -

Sows

NRP_SW_S National Residue Project - Sows -

State

NRP_YC National Residue Project Sampling -

Young Chickens

NRP_YC_S National Residue Project - Young 

Chickens - State

NRP_YT National Residue Project Sampling -

Young Turkeys

NRP_YT_S National Residue Project - Young 

Turkeys - State

PATH_LIVESTK Samples for Histopathological 

Examination - Livestock

Ad hoc samples are collected when a PHV determines that submitting a pathology sample for 

laboratory analysis is needed.

PATH_OTHER Samples for Histopathological 

Examination - Other (non-meat)

PATH_POULTRY Samples for Histopathological 

Examination - Poultry

PATH_PRODUCT Samples for Histopathological 

Examination - Product

RE_CH_CARC01 Religious Exempt Sampling for 

Chicken Carcasses

FSIS collects samples from eligible products 3–4 times per year from eligible establishments (80 

FR 3946). Establishments are eligible for this project if they have been exempted from 

Salmonella verification testing because of a religious exemption (approximately 95 

establishments).

RES_FI Domestic Siluriformes Testing for 

Residue and Speciation

In 2015, FSIS announced (80 FR 79231) it would conduct sampling and testing of Siluriformes

fish and fish products for species and residues to ensure that product is not adulterated or 

misbranded. FSIS has developed a testing project that currently includes the capacity to test for 

malachite green, nitrofurans, veterinary drug residues, gentian violet, metals and pesticides (76 

FR 10439). 

RLMCONT_EL;

RLMCONT_MWL; 

RLMCONT_WL

Risk-based Verification-Midwest LabRTE-Food Contact Surface SampleListeriaThese projects conduct risk-based verification testing to assess the effectiveness of RTE 

operations in controlling L. monocytogenes (69 FR 34221). The projects collect food contact 

surface samples and environmental samples mainly from operations that have a history of 

RLMENVC_EL; problems associated with the proper control for L. monocytogenes.

RLMENVC_MWL; 

RLMENVC_WL

Risk-based Verification-Eastern Lab -

RTE - Non-Food Contact Surface 

Sample-Listeria

RLMENVR_EL; 

RLMENVR_MWL; 

RLMENVR_WL

Risk-based Verification-Eastern LabRTE-Non-Food Contact Brine Sample

RLMPRODC_EL; 

RLMPRODC_MWL; 

RLMPRODC_WL 

Risk-based Verification-Eastern Lab -

RTE - Listeria - Product Sample 

(Composite)

RTEPROD_RAND RTEPROD Sampling - Random RTE 

Products

This sampling project conducts risk-based verification testing to assess the effectiveness of RTE 

operations in controlling L. monocytogenes (69 FR 34221). The agency plans to conduct testing

at modulated frequencies, considering all relevant factors, including the alternative employed 

to address L. monocytogenes, production volume by type of RTE product produced, and the 

establishment's compliance history.

RTEPROD_RISK RTEPROD Sampling - Risk-based RTE 

Products
50

Project Code Sampling Project Name

Purpose and Description of Sampling Project

(from Federal Register Notice Unless Otherwise Indicated)

SPECID Species Identification Sampling Verification of the species claim on domestic products. 

UNKSUB Samples for Identification of 

Unknown Substance

Collector-generated samples primarily for ad hoc identification of extraneous materials 

(Directive 10,230.2)

a Sampling project name is not from PHIS.

b Sampling purpose is not from original Federal Register Notice.

Abbreviations: EL, Eastern Laboratory; MWL, Midwestern Laboratory; PHIS, Public Health Information System; WL, Western Laboratory.
51

Appendix C: ROC Method

The workgroup used the Rank Order Centroid (ROC) method to weight the criteria after the subject 

matter experts had ranked the criteria. The ROC weights are computed using the following formula:

Where Wj is the weight of the j

th item, n is the number of items, and 1/rk is the reciprocal of the ranks 

from j through n. Figure provides an example of converting rankings to weights for five items ranked 1 

to 5. In the example, the weight of the item ranked third out of five would be calculated as w3

(ROC) =

1

5∑ Irk

=

1

5

(

1

3

+

1

4

+

1

5

) = .16

5

k=3

. 

Figure C1. Rank Order Centroid Example with 5 Ranked Items

The occurrence of scoring ties and the variation in responses resulted in a non-linear ranking of the 

criteria. To accommodate the uneven ranking, 1

n

is replaced in the ROC formula by the sum of the 

reciprocal sums 1

(∑ (∑

I

rk

n

k=j

)

n

j=1

)

. The weights for tie rankings are then averaged to compute the final ROC 

weight.

1 Ranks 1 to 5 = (1 +1/2 + 1/3 + 1/4 + 1/5) 0.46

2 Ranks 2 to 5 = (1/2 + 1/3 + 1/4 + 1/5) 0.26

3 Ranks 3 to 5 = (1/3 + 1/4 + 1/5) 0.16

4 Ranks 4 to 5 = (1/4 + 1/5) 0.09

5 Rank 5 = 1/5 0.04

Split the sum into 5 equal parts to get the 

ROC (green bar) Rank Sum reciprocals 

ROC 

Weight
52

Appendix D: Detailed Findings on the Potential Benefits of Ongoing Sampling 

Projects

This appendix presents the detailed scores for the potential benefits that the subject matter experts (SMEs) provided for 

each of FSIS’ sampling projects. These scores were developed based on the criteria the workgroup developed for this 

evaluation (see Box 2 of the main body of this report). 

Table D1 presents the full, raw scoring provided by the SMEs for each question or criteria, with the sampling projects 

listed by rank. Sampling projects for which the criteria apply (that is, that were scored a one) are indicated by a bar 

which is color coded as follows: category 1 criteria (e.g., 1ai) are blue, category 2 criteria are yellow, category 3 criteria 

are green, and category 4 criteria are gold when the criteria apply to the project. The second last column shows the total 

number of criteria that apply to the sampling project, and the dark green bars provide a visual representation of that 

number. The last column shows the overall weighted score for each sampling project, which is represented by the gray 

bars.
53

Table D1: Detailed Responses to Benefits Questions including Weighted/Unweighted Scores by Project

Order Project Name Description of Sampling Project 1ai 1aii 1aiii 1aiv 1av 1bi 1b ii 2a 2b 3a 3b 3c 4a 4b 4c 4d 4e 4f Total Wt Score

1 MT43 Risk-based Sampling of Raw Ground Beef or Veal Products - E.coli O157:H7 & Salmonella 1 1 0 1 0 0 0 0 0 1 0 0 1 1 1 1 0 0 8 0.60

2 MT60 Sampling of Beef Manufacturing Trimmings 1 1 0 1 0 0 0 0 0 1 0 0 1 1 1 1 0 0 8 0.60

3 MT64 Sampling of Raw Ground Beef or Beef Patty Components (other than trim) 1 1 0 1 0 0 0 0 0 1 0 0 1 0 1 1 0 0 7 0.57

4 MT65 Sampling of Bench Trim for further use in ANY raw, non-intact beef products 1 1 0 1 0 0 0 0 0 1 0 0 1 0 1 1 0 0 7 0.57

5 RTEPROD_RAND RTEPROD Sampling - Random RTE Products 1 1 0 1 0 0 0 0 0 1 0 0 1 0 1 1 0 0 7 0.57

6 RTEPROD_RISK RTEPROD Sampling - Risk-based RTE Products 1 1 0 1 0 0 0 0 0 1 0 0 1 0 1 1 0 0 7 0.57

7 INTCONT_LM_E For-Cause Sampling-Eastern Lab-RTE Food Contact Surface-Listeria 1 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 7 0.53

8 INTCONT_LM_M For-Cause Sampling-Midwest Lab-RTE Food Contact Surface-Listeria 1 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 7 0.53

9 INTCONT_LM_W For-Cause Sampling-Western Lab-RTE Food Contact Surface-Listeria 1 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 7 0.53

10 INTCONT_SA_E For-Cause Sampling-Eastern Lab-RTE Food Contact Surface-Salmonella1 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 7 0.53

11 INTCONT_SA_M For-Cause Sampling-Midwest Lab-RTE Food Contact Surface-Salmonella1 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 7 0.53

12 INTPROD_LM_E For-Cause Sampling-Eastern Lab-RTE Product-Listeria 1 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 7 0.53

13 INTPROD_LM_M For-Cause Sampling-Midwest Lab-RTE Product-Listeria 1 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 7 0.53

14 INTPROD_LM_W For-Cause Sampling-Western Lab-RTE Product-Listeria 1 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 7 0.53

15 INTPROD_SA_E For-Cause Sampling-Eastern Lab-RTE Product-Salmonella 1 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 7 0.53

16 INTPROD_SA_M For-Cause Sampling-Midwest Lab-RTE Product-Salmonella 1 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 7 0.53

17 MT44T Follow-up E.coli Sampling of Raw Ground Beef, Trimmings or Components (Traceback) 1 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 7 0.53

18 MT05 Raw Ground Beef or Veal Sampling - Retail 1 1 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 6 0.51

19 HC_CH_CARC01 HACCP Verification for Young Chicken Carcasses 0 1 1 0 0 0 0 0 0 1 0 1 1 1 1 1 0 0 8 0.52

20 HC_CH_COM01 Sampling for Ground and Other Comminuted Chicken (not Mechanically Separated) 0 1 1 0 0 0 0 0 0 1 0 1 1 1 1 1 0 0 8 0.52

21 HC_CPT_LBW01 Sampling for Chicken Parts – Legs, Breasts, and Wings 0 1 1 0 0 0 0 0 0 1 0 1 1 1 1 1 0 0 8 0.52

22 HC_TU_CARC01 HACCP Verification for Young Turkey Carcasses 0 1 1 0 0 0 0 0 0 1 0 1 1 1 1 1 0 0 8 0.52

23 HC_TU_COM01 Sampling for Ground and Other Comminuted Turkey (not Mechanically Separated) 0 1 1 0 0 0 0 0 0 1 0 1 1 1 1 1 0 0 8 0.52

24 COMPLIAN Investigative Sampling 1 1 0 1 0 0 1 1 1 0 0 0 0 0 1 0 0 0 7 0.46

25 RLMCONT_EL Risk-based Verification-Eastern Lab - RTE - Food Contact Surface Sample - Listeria 1 0 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 5 0.46

26 RLMCONT_MWL Risk-based Verification-Midwest Lab-RTE-Food Contact Surface Sample-Listeria 1 0 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 5 0.46

27 RLMCONT_WL Risk-based Verification-Western Lab-Food Contact Surface Sample-Listeria 1 0 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 5 0.46

28 RLMPRODC_EL Risk-based Verification-Eastern Lab - RTE - Listeria - Product Sample (Composite) 1 0 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 5 0.46

29 RLMPRODC_MWL Risk-based Verification-Midwestern Lab - RTE - Listeria - Product Sample (Composite) 1 0 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 5 0.46

30 RLMPRODC_WL Risk-based Verification-Western Lab - RTE - Listeria - Product Sample (Composite) 1 0 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 5 0.46

31 MT06 Raw Ground Beef or Veal Sampling - Retail - Follow-up 1 1 0 1 0 0 0 0 1 1 0 0 0 0 1 0 0 0 6 0.41

32 MT44 Follow-up Risk-based Sampling of Positive Raw Ground Beef or Veal Sample -E.coli O157:H7/Salmonella 1 1 0 1 0 0 0 0 1 1 0 0 0 0 1 0 0 0 6 0.41

33 MT52 Follow-up Sampling of Suppliers of Raw Ground Beef Trim or Components 1 1 0 1 0 0 0 0 1 1 0 0 0 0 1 0 0 0 6 0.41

34 MT53 Follow-up Sampling of Beef Mfg Trim or Other Raw Ground Beef or Beef Patty Components 1 1 0 1 0 0 0 0 1 1 0 0 0 0 1 0 0 0 6 0.41

35 NRP_FS National Residue Program Sampling - Feral Swine 1 0 0 1 0 0 0 0 0 1 0 1 1 0 0 0 1 0 6 0.50

36 EM31 Egg Product Sampling-Pasteurized-Egg Whites-Salmonella and Lm 1 0 0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0 5 0.40

37 EM32 Egg Product Sampling-Pasteurized-Whole Egg or Yolks - Salmonella and Lm 1 0 0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0 5 0.40

38 EM33 Egg Product Sampling-Pasteurized-Whole Eggs with Added Yolks or Whole Egg Blends-Salmonella and Lm 1 0 0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0 5 0.40

39 EM34 Egg Product Sampling-Pasteurized-Whole Eggs or Yolks with > 2% salt or sugar added-Salmonella and Lm 1 0 0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0 5 0.40

40 EM35 Egg Product Sampling-Pasteurized-Dried Yellow Egg Products - Salmonella and Lm 1 0 0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0 5 0.40

41 EM36 Egg Product Sampling-Pasteurized-Dried Egg Whites - Salmonella and Lm1 0 0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0 5 0.40

42 EM37 Egg Product Sampling-Pasteurized-Pan Dried Egg Whites - Salmonella and Lm 1 0 0 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0 5 0.40

43 KIS KIS - Samples from In-plant Testing 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

44 NRP_BC National Residue Program Sampling - Beef Cows 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

45 NRP_BS National Residue Program Sampling - Bull/Stag 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

46 NRP_BV National Residue Program Sampling - Bob Veal 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

47 NRP_DC National Residue Program Sampling - Dairy Cows 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

48 NRP_FFV National Residue Program Sampling - Formula-Fed Veal 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

49 NRP_GO National Residue Program Sampling - Goats 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

50 NRP_HC National Residue Program Sampling - Heavy Calf 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

51 NRP_HF National Residue Program Sampling - Heifers 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

52 NRP_MS National Residue Program Sampling - Market Swine 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

53 NRP_NFFV National Residue Program Sampling - Non Formula-Fed Veal 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

54 NRP_OBT National Residue Program - Old Breeder Turkeys 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

55 NRP_RS National Residue Sampling Program - Roaster Swine 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48
54

Order Project Name Description of Sampling Project 1ai 1aii 1aiii 1aiv 1av 1bi 1b ii 2a 2b 3a 3b 3c 4a 4b 4c 4d 4e 4f Total Wt Score

56 NRP_SH National Residue Program Sampling - Sheep 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

57 NRP_ST National Residue Program Sampling - Steers 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

58 NRP_SW National Residue Program Sampling - Sows 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

59 NRP_YC National Residue Program Sampling - Young Chickens 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

60 NRP_YT National Residue Program Sampling - Young Turkeys 1 0 0 1 0 0 0 0 0 1 0 0 1 0 0 0 1 0 5 0.48

61 FRTESALMONEL Import-Follow-up to Salmonella Positive in RTE Product 1 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 0 4 0.39

62 EGGIMP Import-Egg Products-Salmonella and Lm 1 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 4 0.39

63 IMVRTE Import-Micro Pathogen Sampling of RTE Products 1 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 4 0.39

64 MT08 Import-Sampling of Raw Ground or Communited Beef or Veal Product1 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 4 0.39

65 MT51 Import-Raw Beef Manufactured Trimmings or Components for use in Ground Beef or Beef Products 1 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 4 0.39

66 INTENV_LM_E For-Cause Sampling-Eastern Lab-Non-Food Surface-Listeria 0 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 6 0.38

67 INTENV_LM_M For-Cause Sampling-Midwest Lab-Non-Food Surface-Listeria 0 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 6 0.38

68 INTENV_LM_W For-Cause Sampling-Western Lab-Non-Food Surface-Listeria 0 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 6 0.38

69 INTENV_SA_E For-Cause Sampling-Eastern Lab-Non-Food Surface-Salmonella 0 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 6 0.38

70 INTENV_SA_M For-Cause Sampling-Midwest Lab-Non-Food Surface-Salmonella 0 1 0 1 0 0 0 1 1 1 0 0 0 0 1 0 0 0 6 0.38

71 CG_RES_EL Collector Generated - Residue - Eastern Lab 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

72 CG_RES_MWL Collector Generated - Residue - Midwestern Lab 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

73 CG_RES_WL Collector Generated - Residue - Western Lab 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

74 CG_SHOW_MWL Collector Generated-Residues-Show Animals-Midwest Lab 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

75 CG_SHOW_WL Collector Generated-Residue-Show Animals-Western Lab 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

76 NRP_BC_S National Residue Program - Beef Cows - State 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

77 NRP_BV_S National Residue Program - Bob Veal - State 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

78 NRP_DC_S National Residue Program - Dairy Cows - State 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

79 NRP_HF_S National Residue Program - Heifers - State 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

80 NRP_MS_S National Residue Program - Market Swine - State 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

81 NRP_ST_S National Residue Program - Steers - State 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

82 NRP_SW_S National Residue Program - Sows - State 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

83 NRP_YC_S National Residue Program - Young Chickens - State 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

84 NRP_YT_S National Residue Program - Young Turkeys - State 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 4 0.40

85 IMPABNCONT Import-Abnormal Container 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 3 0.31

86 IMPAMRBEEF Import - AMR Product - Beef 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 3 0.31

87 IMPFISH_CH_E Import Siluriformes Chemistry Sampling - Eastern Laboratory 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 3 0.31

88 IMPFISH_CH_W Import Siluriformes Chemistry Sampling - Western Laboratory 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 3 0.31

89 F_CH_CARC01 Follow-up sampling of Chicken Carcasses 0 1 0 0 0 0 0 1 0 1 0 0 0 0 1 0 0 0 4 0.32

90 F_TU_CARC01 Follow-up sampling of Turkey Carcasses 0 1 0 0 0 0 0 1 0 1 0 0 0 0 1 0 0 0 4 0.32

91 FLISTERIA Import-Follow-up to Listeria monocytogenes Positive 1 0 0 1 0 0 0 0 1 0 0 0 0 0 1 0 0 0 4 0.29

92 RLMENVC_EL Risk-based Verification-Eastern Lab - RTE - Non-Food Contact Surface Sample-Listeria 0 0 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 4 0.31

93 RLMENVC_MWL Risk-based Verification-Midwest Lab-RTE-Non-Food Contact Surface Sample-Listeria 0 0 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 4 0.31

94 RLMENVC_WL Risk-based Verification-Western Lab-RTE-Non-food Contact Surface Sample-Listeria 0 0 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 4 0.31

95 RLMENVR_EL Risk-based Verification-Eastern Lab-RTE-Non-food Contact Brine Sample 0 0 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 4 0.31

96 RLMENVR_MWL Risk-based Verification-Midwest Lab-RTE-Non-food Contact Brine Sample 0 0 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 4 0.31

97 RLMENVR_WL Risk-based Verification-Western Lab-RTE-Non-food Contact Brine Sample 0 0 0 1 0 0 0 0 0 1 0 0 1 0 1 0 0 0 4 0.31

98 RES_FI Domestic Siluriformes Testing for Residue and Speciation 1 0 0 1 0 0 1 0 0 1 0 0 0 0 0 0 1 0 5 0.37

99 AMS_PROD_RTE AMS RTE Canada EV Program - Product Collected by FSIS 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 3 0.32

100 EXP_CH_MSK01 Exploratory Sampling for Mechanically Separated Chicken 0 0 0 0 0 0 0 0 0 1 0 1 0 0 1 1 0 0 4 0.23

101 EXP_CPT_OT01 Exploratory Sampling for Chicken Parts - Others Parts 0 0 0 0 0 0 0 0 0 1 0 1 0 0 1 1 0 0 4 0.23

102 EXP_CPT_QH01 Exploratory Sampling for Chicken Parts - Quarter and Half Carcasses 0 0 0 0 0 0 0 0 0 1 0 1 0 0 1 1 0 0 4 0.23

103 EXP_PK_COM02 Baseline Sampling for Pork - Comminuted (Ground, Mechanically Separated, and Other Comminuted) 0 0 0 0 0 0 0 0 0 1 0 1 0 0 1 1 0 0 4 0.23

104 EXP_PK_ICT02 Baseline Sampling for Pork - Intact cuts 0 0 0 0 0 0 0 0 0 1 0 1 0 0 1 1 0 0 4 0.23

105 EXP_PK_NCT02 Baseline Sampling for Pork - Non-Intact Cuts 0 0 0 0 0 0 0 0 0 1 0 1 0 0 1 1 0 0 4 0.23

106 EXP_TU_MSK01 Exploratory Sampling for Mechanically Separated Turkey 0 0 0 0 0 0 0 0 0 1 0 1 0 0 1 1 0 0 4 0.23

107 IMP_PORK Imported Raw Pork Product Sampling 0 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 3 0.23

108 IMP_POULTRY Import Sampling of Raw and NRTE Poultry Products 0 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 3 0.23

109 IMPFISH_MI Import Siluriformes Microbiology Sampling 0 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 3 0.23

110 IMPHORMONES Import Hormones 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0.19
55
56

Appendix E: Required Number of Samples Estimates for Published Pathogen 

Rates

FSIS posts pathogen rates for select sampling projects quarterly. Reported estimates are either prevalence, volumeweighted percent positive, or straight percent positive. This appendix uses values from CY2018 published values because 

it has the most current pathogen rates to demonstrate the number of samples required for confidence interval 

estimates and to determine a difference between two proportions. 

The number of samples required to meet the maximum margin of error for the estimated pathogen rate is based off the 

requirement discussed in Phase 3 on Table 5: Parameters for a standard maximum margin of error of a confidence 

interval for an estimate of a pathogen rate . 

The number of required samples for these projects tends to be much higher than the number of samples allocated in the 

Annual Sampling Plan. In a few instances, the number of samples required is much lower. The projects where the 

required number of samples is considerably lower than the Annual Sampling Plan are highlighted.

The number of samples required for each proportion is calculated from either the current estimated pathogen rate, or 

from the actual FSIS documentation. For current estimated rates, the goal listed is a 25% pathogen (the stated Healthy 

People 2020 goal) rate reduction. For projects with specific goals, the number of samples required is based on the target 

pathogen rate, not the current estimated pathogen rate.
57

Table E1: Estimated Number of Samples Required for Projects with Published Pathogen Rates

1 – Prevalence 2 – Volume Weighted Percent Positive 3 – Percent Positive 4 – Number of samples required to detect the difference officially stated by FSIS

Workgroup Project Pathogen

FY 2019

Annual 

Sampling 

Plan

Interval Estimates Difference Between Two Proportions

Estimated 

Pathogen 

Rate CY 2018

Max 

Margin 

of Error

Significance 

Level 

(alpha)

Required 

# of 

Samples 

Initial 

Pathogen 

Rate

Target 

Pathogen 

Rate

Required # of 

Samples for Each 

Proportion 

SCCG HC_CH_CARC01 Salmonella 9,000 4.49%1 1% 0.05 1,791 7.50% 5.63% 2,7354

Campylobacter 13.03%3 1% 0.05 4,354

HC_CH_LBW01 Salmonella 9,000 12.99%1 1% 0.05 4,342 28% 18% 2754

Campylobacter 15.40%3 1% 0.05 5,005

HC_CH_COM01 Salmonella 2,500 38.32%1 1% 0.05 9,080 49% 34% 1664

Campylobacter 7.96%3 1% 0.05 2,815

HC_TU_CARC01 Salmonella 2,000 0.53%1 0.53% 0.05 721 1.70% 1.275% 12,7324

Campylobacter 0.51%3 0.51% 0.05 750

HC_TU_COM01 Salmonella 1,500 23.21%1 1% 0.05 6,847 19.90% 14% 6314

Campylobacter 2.53%3 1% 0.05 948 1.20% 1.0% 42,6914

EXP_CH_MSK01 Salmonella 150 80.95%3 5% 0.05 237 80.95% 60.71% 76

Campylobacter 62.50%3 5% 0.05 361 62.50% 46.88% 156

EXP_TU_MSK01 Salmonella 150 48.48%3 1% 0.05 9,595 48.48% 36.36% 258

Campylobacter 9.09%3 1% 0.05 3,175 9.09% 6.82% 2,227

EXP_CPT_QH01 Salmonella 120 15.60%3 1% 0.05 5,058 15.60% 11.70% 1,213

Campylobacter 20%3 1% 0.05 6,147 20% 15% 903

EXP_CPT_OT01 Salmonella 360 54.15%3 5% 0.05 382 54.15% 40.62% 210

STEC MT43 E. coli O157:H7 11,500 0.01%1 0.01% 0.05 38,411 

Salmonella 3.36%1 1.00% 0.05 1,248 3.36% 2.52% 6,345

MT60 E. coli O157:H7 3,750 0.15%1 0.15% 0.05 2,558

non-O157 STEC 0.25%1 0.25% 0.05 1,533

Salmonella 1.86%1 1.00% 0.05 702 1.86% 1.39% 11,620

MT64 E. coli O157:H7 1,050 0.17%3 0.24% 0.05 2,256

Salmonella 7.11%3 1.00% 0.05 2,538 7.11% 5.33% 2,895

MT65 E. coli O157:H7 1,500 0.00%3--% 0.05 --

Salmonella 0.58%3 0.58% 0.05 659 0.58% 0.44% 37,695

Pork EXP_PK_ICT02 Salmonella 1,521 10.57%3 1.00% 0.05 3,632 10.57% 7.93% 1,883

EXP_PK_NCT02 Salmonella 1,272 7.15%3 1.00% 0.05 2,551 7.15% 5.37% 2,878

EXP_PK_COM02 Salmonella 1,704 20.88%3 1.00% 0.05 6,347 20.88% 15.66% 857

Egg 

Products 

EM Salmonella 1,600 0.00%2--% 0.05 --

L. monocytogenes 0.00%2--% 0.05 --

RTE RTEPROD_RAND Salmonella 7,400 0.01%2 0.01% 0.05 38,411

L. monocytogenes 0.10%2 0.10% 0.05 3,838

RTEPROD_RISK Salmonella 7,400 0.02%2 0.02% 0.05 19,204

L. monocytogenes 0.01%2 0.01% 0.05 38,411
58

Appendix F: Summary, by sampling project, of analyses conducted using results from each project

Table F1 documents how the Agency has used the data and what actions they are able to perform with the data. It also indicates how the Agency reports results.

Information presented in this table were current during the time of the evaluation. The number of planned samples (for FY18 and FY19) and actual samples are 

included. The ‘N/A’ value for planned samples designates that the Agency cannot plan for these types of samples due to their nature. They are either collector 

generated, investigative or follow-up sampling which has been done on an ad hoc basis dependent upon the need. Many individual establishment specific 

sample results are posted on the website but are not reflected in this chart. These results can be found at 

https://www.fsis.usda.gov/wps/portal/fsis/topics/data-collection-and-reports/data.

Table F1: Analyses of data by project 

Project Code Sampling Project Name Analyses Conducted and Decisions Based 

on Results from Sampling Project Results Reporting

# Samples 

FY18 Sampling 

Plan

# Samples 

Analyzed 

FY18

# Samples FY19 

Sampling Plan

ABNCONT Abnormal Containera

• OFO and OIEA collect ad hoc samples based on their 

observations.

• Individual results can lead to regulatory action.

• No overall surveillance 

or reports.

N/A 0 N/A

AMR01 Advanced Meat Recovery Product

• Conducted ad hoc.

• Individual results can lead to regulatory action.

• Per Directive 7160.3, data are reviewed annually to 

determine effectiveness of the directive.

• No overall surveillance 

or reports.

150 101 150

AMS_PROD_RTE AMS RTE Canada EV Project - Product 

Collected by FSIS

•Results used for export certificates as part of Export 

Verification program. 

• No overall surveillance.

• Included in Quarterly 

Establishment Letters.

N/A N/A N/A

CG_RES_EL; 

CG_RES_MWL; 

CG_RES_WL

Collector Generated - Residue -

Eastern Lab/ Midwestern Lab/ 

Western Lab

• Chemical Residue workgroup analyzes results to 

inform guidance to IPP on which animals to target.

• Residue sampling results 

are published annually in 

the Red Book

• Included in Quarterly 

Establishment Letters.

N/A 197 N/A

CG_SHOW_MWL; 

CG_SHOW_WL

Collector Generated-Residues-Show 

Animals-Midwest Lab/Western Lab

• Residue testing results inform future allocations of 

residue testing.

• Violative results are shared with FDA for FDA 

regulatory action. 

• Residue sampling results 

are published annually in 

the Red Book.

• Included in Quarterly 

Establishment Letters.

N/A 82 N/A

COMPLIAN Investigative Sampling

• OIEA conducts ad hoc testing. 

• Individual results can lead to regulatory action.

• Results from investigations have informed policy for 

specific establishments, outreach, and guidelines.

• Supports recalls and investigations and is used as 

evidence in court cases.

• No overall surveillance 

or reports.

N/A 151 N/A

EGGIMP Import-Egg Products-Salmonella and 

Lm

• Individual results can lead to regulatory action.

• Results used to verify ongoing country equivalence.

• Import refusals are 

posted to FSIS website. 150 104 150
59

Project Code Sampling Project Name Analyses Conducted and Decisions Based 

on Results from Sampling Project Results Reporting

# Samples 

FY18 Sampling 

Plan

# Samples 

Analyzed 

FY18

# Samples FY19 

Sampling Plan

EM31 Egg Product Sampling-PasteurizedEgg Whites-Salmonella and Lm

• Sampled lot of product shipped to non-official plant 

must be held pending negative test results. 

• Sampling data posted on FSIS website.

• Positive test results require plants to conduct 

investigation on root cause and implement 

corrective actions.

• Product that tests positive cannot enter commerce, 

but it can be reprocessed and must pass all parts of 

9 CFR 590 prior to release..

• Aggregate data posted 

on FSIS website. 1,600 1,693 1,600

EM32

Egg Product Sampling-PasteurizedWhole Egg or Yolks - Salmonella and 

Lm

EM33

Egg Product Sampling-PasteurizedWhole Eggs with Added Yolks or 

Whole Egg Blends-Salmonella and Lm

EM34

Egg Product Sampling-PasteurizedWhole Eggs or Yolks with > 2% salt or 

sugar added-Salmonella and Lm

EM35

Egg Product Sampling-PasteurizedDried Yellow Egg Products -

Salmonella and Lm

EM36 Egg Product Sampling-PasteurizedDried Egg Whites - Salmonella and Lm

EM37

Egg Product Sampling-PasteurizedPan Dried Egg Whites - Salmonella 

and Lm

EXP_CH_MSK01 Exploratory Sampling for 

Mechanically Separated Chicken

* The results of this sampling project have been 

analyzed on multiple occasions. This product can be 

used in comminuted product, which is under a 

performance standard. A decision for the mechanically 

separated poultry projects is pending the results of an 

establishment survey regarding product usage.

• No overall surveillance.

• Aggregate data posted 

on FSIS website.

• Included in Quarterly 

Establishment Letters

150 105 150

EXP_CPT_OT01 Exploratory Sampling for Chicken 

Parts – Other Parts

* As a result of analysis of the data and particularly 

high pathogen positive rates in chicken livers, FSIS

developed a Chicken Liver Action Plan 

* The SCCG has compared these sampling results with 

that of LBW (legs, breasts, wings). In FY 2020, FSIS will 

consider having Quarters and Halves fall under the 

LBW sampling project. This would provide additional 

opportunities for establishments to be categorized 

(more product for IPP to select from) and prevent 

establishments from escaping performance standards 

and/or sampling by altering their cut of the bird.

• Aggregate data posted 

on FSIS website.

• Included in Quarterly 

Establishment Letters

360 332 360

EXP_CPT_QH01 Exploratory Sampling for Chicken 

Parts - Quarter and Half Carcasses 120 109 120
60

Project Code Sampling Project Name Analyses Conducted and Decisions Based 

on Results from Sampling Project Results Reporting

# Samples 

FY18 Sampling 

Plan

# Samples 

Analyzed 

FY18

# Samples FY19 

Sampling Plan

EXP_FI_MIC01 Domestic Siluriformes Sampling for 

Microbiology

* The domestic Siluriformes data, particularly that for 

Salmonella, is continually analyzed to determine what 

kind of risk such fish poses. 

* There is indication that Salmonella in these fish 

demonstrate seasonality, but more data is needed to 

confirm.

• No overall surveillance. 650 611 650

EXP_LV_NUTR Label Verification for Sodium and Fat 

Content

* The data is used to inform policy development for 

product labeling, future sampling, to verify that the 

labels are truthful and not misleading (317.8 and 

381.129) and to verify product is not adulterated.

* Results from previous analyses have been used to 

inform establishments when they are not following the 

label verification guidelines, such as the nutrition facts 

panel testing.

• No overall surveillance.

200 121 200

EXP_LV_SOY Label Verification for Soy 200 12 200

EXP_LV_ABX Label Verification for Antibiotics 400 70 400

EXP_LV_HORM Label Verification for Hormone Free 200 23 200

EXP_PK_COM02

Baseline Sampling for Pork -

Comminuted (Ground, Mechanically 

Separated, and Other Comminuted) • FSIS is currently using the results from these 

sampling projects to develop performance standards 

for pork products. 

• Aggregate data posted 

on FSIS website.

• Included in Quarterly 

Establishment Letters

1,120 1,665 1,704

EXP_PK_ICT02 Baseline Sampling for Pork - Intact 

cuts

840 1,315 1,521

EXP_PK_NCT02 Baseline Sampling for Pork - NonIntact Cuts 840 1,193 1,272

EXP_TU_MSK01 Exploratory Sampling for 

Mechanically Separated Turkey

* The results of this sampling project have been 

analyzed on multiple occasions. This product can be 

used in comminuted product, which is under a 

performance standard. A decision for the mechanically 

separated poultry projects is pending the results of an 

establishment survey regarding product usage.

• Aggregate data posted 

on FSIS.

• No overall surveillance.

• Included in Quarterly 

Establishment Letters

150 99 150

F_CH_CARC01 Follow-up sampling of Chicken 

Carcasses

• Results of follow-up sampling inform PHRE evaluation 

and to determine effectiveness of corrective actions 

implemented by establishments in response to not 

meeting a performance standard.

• No overall surveillance.

• Included in Quarterly 

Establishment Letters

N/A 2,634 N/A

F_TU_CARC01 Follow-up sampling of Turkey 

Carcasses

F_CPT_LBW01 Follow-up sampling of Chicken Parts

F_TU_COM01 Follow-up sampling of Comminuted 

Turkey

F_CH_COM01 Follow-up sampling of Comminuted 

Chicken
61

Project Code Sampling Project Name Analyses Conducted and Decisions Based 

on Results from Sampling Project Results Reporting

# Samples 

FY18 Sampling 

Plan

# Samples 

Analyzed 

FY18

# Samples FY19 

Sampling Plan

FAMR01 Follow-up Sampling to Advanced 

Meat Recovery Positive

• Per Directive 7160.3, data are reviewed annually to 

determine effectiveness of the directive.

• Individual results can lead to regulatory action.

• No overall surveillance 

or reports.

N/A 40 N/A

FDS01 Food Defense Sampling - Meals 

Ready-to-Eat

• No overall surveillance

or reports.

N/A 14 N/A

FDS05 Food Defense Sampling- Inauguration 

Testing

• No overall surveillance

or reports.

N/A N/A N/A

FLISTERIA Import-Follow-up to Listeria 

monocytogenes Positive

• Individual results can lead to regulatory action.

• Results used to verify ongoing county equivalence

• Import refusals are 

posted to FSIS website. N/A 34 N/A

FOODCHEM Collector-Generated Food Chemistry 

Samples • Individual results can lead to regulatory action. • No overall surveillance or reports.

N/A 3 N/A

FRTESALMONEL Import-Follow-up to Salmonella 

Positive in RTE Product

• Individual results can lead to regulatory action.

• Results used to verify ongoing county equivalence.

• Import refusals are 

posted to FSIS website. N/A 0 N/A

HC_CH_CARC01 HACCP Verification for Young Chicken 

Carcasses

• Sampling results are used to verify process control 

through verification of meeting a performance 

standard.

• Establishments are categorized based on sample 

results. 

• Establishment categories are posted on FSIS website.

• Sampling results are used to calculate prevalence.

• Sampling results have set a Salmonella performance 

standard for industry.

• Aggregate data posted 

on FSIS website

• Individual establishment 

category status is posted 

on the FSIS website.

• Included in Quarterly 

Establishment Letters

10,000 9,207 9,000

HC_TU_CARC01 HACCP Verification for Turkey 

Carcasses 2,000 1,936 2,000

HC_CH_COM01

Sampling for Ground and Other 

Comminuted Chicken (not 

Mechanically Separated)

2,500 1,895 2,500

HC_TU_COM01

Sampling for Ground and Other 

Comminuted Turkey (not 

Mechanically Separated)

1,500 1,467 1,500

HC_CPT_LBW01 Sampling for Chicken Parts – Legs,

Breasts, and Wings 8,000 7,860 9,000

IMP_PORK Imported Raw Pork Product Sampling • Import refusals are 

posted to FSIS website. 1,000 384 900

IMP_POULTRY Import Sampling of Raw and NRTE 

Poultry Products

• Import refusals are 

posted to FSIS website. 1,000 551 900

IMPABNCONT Import - Abnormal Container • Individual results can lead to regulatory action. • No overall surveillance 

or reports.

N/A 10 N/A
62

Project Code Sampling Project Name Analyses Conducted and Decisions Based 

on Results from Sampling Project Results Reporting

# Samples 

FY18 Sampling 

Plan

# Samples 

Analyzed 

FY18

# Samples FY19 

Sampling Plan

IMPAMRBEEF Import - AMR Product - Beef

• Individual results can lead to regulatory action.

• Per Directive 7160.3, data are reviewed annually to 

determine effectiveness of the directive.

• Results used to verify ongoing county equivalence.

• Import refusals are 

posted to FSIS website. 10 0 10

IMPFISH_CH_E; 

IMPFISH_CH_W

Import Siluriformes Chemistry 

Sampling - Eastern 

Laboratory/Western Laboratory

• Individual results can lead to regulatory action.

• Results used to verify ongoing county equivalence.

• Import refusals are 

posted to FSIS website. 1,800 582 2,000

IMPFISH_MI Import Siluriformes Microbiology 

Sampling

• The import Siluriformes data is continuously analyzed 

to determine what kind of risk Siluriformes poses. 

• Individual results can lead to regulatory action.

• Results used to verify ongoing county equivalence

• Import refusals are 

posted to FSIS website 1,800 223 1,000

IMPMETALS National Residue Plan - Import 

Sampling - Metals - Eastern Lab

• Individual results can lead to regulatory action.

• Results used to verify ongoing county equivalence.

• Import refusals are 

posted to FSIS website.

3,000 2,823 2,000

IMPPESTICIDE National Residue Plan - Import 

Sampling - Pesticides - Western Lab

IMPRESEGG Import Residue Sampling - Egg 

Products

IMPRESFR_EL; 

IMPRESFR_WL

National Residue Plan - Import 

Sampling - Fresh Product

IMPRESPR_EL; 

IMPRESPR_MWL

National Residue Plan - Import 

Sampling - Processed Products

IMPSPECIESID Import-Species Identification 250 155 250

IMVRTE Import-Micro Pathogen Sampling of 

RTE Products 3,000 2,942 3,000

INTCONT_LM_E; 

INTCONT_LM_M; 

INTCONT_LM_W

For-Cause Sampling- RTE Food 

Contact Surface-Listeria

• Conducted after positive results from 

RTEPROD_RAND, RTEPROD_RISK sampling project.

• Individual results can lead to regulatory action.

• Verification of 9 CFR 430 requirements to control 

Listeria monocytogenes in post-lethality RTE products 

and food contact surfaces.

• Evaluation of potential cross-contamination or 

harborage of Listeria monocytogenes

• Results of follow-up sampling inform PHRE 

evaluation and to determine effectiveness of 

corrective actions implemented by establishments in 

response to a previous positive.

• No overall surveillance.

• Included in Quarterly 

Establishment Letters

N/A 970 N/A

INTCONT_SA_E; 

INTCONT_SA_M

For-Cause Sampling- RTE Food 

Contact Surface-Salmonella

INTENV_LM_E; 

INTENV_LM_M; 

INTENV_LM_W

For-Cause Sampling- Non-Food 

Surface-Listeria

N/A 543 N/A

INTENV_SA_E; 

INTENV_SA_M

For-Cause Sampling- Non-Food 

Surface-Salmonella
63

Project Code Sampling Project Name Analyses Conducted and Decisions Based 

on Results from Sampling Project Results Reporting

# Samples 

FY18 Sampling 

Plan

# Samples 

Analyzed 

FY18

# Samples FY19 

Sampling Plan

INTPROD_LM_E; 

INTPROD_LM_M; 

INTPROD_LM_W

For-Cause Sampling- RTE ProductListeria

N/A 480 N/A

INTPROD_SA_E; 

INTPROD_SA_M

For-Cause Sampling-RTE ProductSalmonella

KIS KIS - Samples from In-plant Testing

• Used to inform policy changes about inspectorgenerated sampling and NRP.

• KIS tests conducted ad hoc when a PHV suspects use 

of antibiotics in livestock. Positive KIS tests are sent to 

the labs for verification.

• KIS tests conducted ad 

hoc. Positive KIS tests are 

sent to the labs for 

verification.

N/A 3,841 N/A

RE_CH_CARC01 Religious Exempt Sampling for 

Chicken Carcasses

• Data are being analyzed to determine Agency next 

steps.

• Aggregate data posted 

on FSIS.

• No overall surveillance.

• Included in Quarterly 

Establishment Letters

6,600

360

2,200

LO_CH_CARC01 Very Low Volume Sampling for 

Chicken Carcasses

2,005

LO_CH_COM01

Very Low Volume Sampling for 

Ground and Other Comminuted 

Chicken (not Mechanically Separated)

LO_CPT_LBW01 Very Low Volume Sampling for Raw 

Chicken Parts - Legs, Breast, Wings

LO_CPT_OT01 Very Low Volume Sampling for Raw 

Chicken Parts - Other Parts

LO_CPT_QH01 Very Low Volume Sampling for 

Chicken Parts - Quarters/Halves

LO_TU_CARC01 Very Low Volume Sampling for Turkey 

Carcasses

LO_TU_COM01

Very Low Volume Sampling for 

Ground and Other Comminuted 

Turkey (not Mechanically Separated)

LO_CH_MSK01 Very Low Volume Sampling for 

Mechanically Separated Chicken

LO_TU_MSK01 Very Low Volume Sampling for 

Mechanically Separated Turkey

MT05 Raw Ground Beef or Veal Sampling -

Retail

• Individual results can lead to regulatory action.

• The sample questionnaire includes questions 

regarding the Grinding Records Rule to evaluate 

compliance with recordkeeping requirements to 

• Results are posted on 

the FSIS website. 575 557 575
64

Project Code Sampling Project Name Analyses Conducted and Decisions Based 

on Results from Sampling Project Results Reporting

# Samples 

FY18 Sampling 

Plan

# Samples 

Analyzed 

FY18

# Samples FY19 

Sampling Plan

ensure trace back, which is evaluated annually to 

inform outreach to retailers.

MT06 Raw Ground Beef or Veal Sampling -

Retail - Follow-up

• Individual results can lead to regulatory action. N/A 0 N/A

MT08 Import-Sampling of Raw Ground or 

Comminuted Beef or Veal Product

• Individual results can lead to regulatory action.

• Results used to verify ongoing country equivalence.

• Import refusals are 

posted to FSIS website. 50 35 50

MT44

Follow-up Risk-based Sampling of 

Positive Raw Ground Beef or Veal 

Sample -E. coli O157:H7/ Salmonella

• Individual results can lead to regulatory action.

Results of follow-up sampling inform PHRE evaluation 

and to determine effectiveness of corrective actions 

implemented by establishments in response to 

producing adulterated product (for E. coli)

• Results are posted on 

the FSIS website.

• Included in Quarterly 

Establishment Letters

N/A 50 N/A

MT44T

Follow-up E. coli Sampling of Raw 

Ground Beef, Trimmings or 

Components (Traceback)

N/A 64 N/A

MT51

Import-Raw Beef Manufactured 

Trimmings or Components for use in 

Ground Beef or Beef Products

• Individual results can lead to regulatory action.

• Results used to verify ongoing county equivalence.

• Import refusals are 

posted to FSIS website. 2,000 1,003 1,500

MT52

Follow-up Sampling of Suppliers of 

Raw Ground Beef Trim or 

Components

• Individual results can lead to regulatory action.

• Results of routine sampling inform PHRE evaluation 

and to determine effectiveness of STEC and 

Salmonella controls implemented by 

establishments.

• Results are posted on 

the FSIS website.

• Included in Quarterly 

Establishment Letters

N/A 59 N/A

MT53

Follow-up Sampling of Beef Mfg. Trim 

or Other Raw Ground Beef or Beef 

Patty Components

N/A 601 N/A

MT43

Risk-based Sampling of Raw Ground 

Beef or Veal Products - E. coli 

O157:H7 & Salmonella • Aggregate data posted 

on FSIS website.

• Results are posted on 

the FSIS website.

• Included in Quarterly 

Establishment Letters

11,500 11,035 11,500

MT60 Sampling of Beef Manufacturing 

Trimmings 3,750 3,737 3,750

MT64 Sampling of Raw Ground Beef or Beef 

Patty Components (other than trim) 600 1,170 1,050

MT65

Sampling of Bench Trim for further 

use in ANY raw, non-intact beef 

products

1,500 1,206 1,500

NARMS_BC NARMS-Sampling-Beef Cows • Data is incorporated into 

a NARMS integrated 

report, which is a 

combined analysis of 

FDA/CDC/FSIS NARMSrelated test results with a 

6,400 6,486 6,400 NARMS_DC NARMS- Sampling-Dairy Cows (Paired 

with RESNARMS_DC)

NARMS_HF NARMS- Sampling-Heifers
65

Project Code Sampling Project Name Analyses Conducted and Decisions Based 

on Results from Sampling Project Results Reporting

# Samples 

FY18 Sampling 

Plan

# Samples 

Analyzed 

FY18

# Samples FY19 

Sampling Plan

NARMS_MS NARMS- Sampling-Market Swine focus on trends in 

antimicrobial resistance.

NARMS_ST NARMS- Sampling-Steers

NARMS_SW NARMS- Sampling-Sows

NARMS_YC NARMS- Sampling-Young Chickens 

(Paired with RESNARMS_YC)

NARMS_YT NARMS- Sampling-Young Turkeys

NRP_BC_S National Residue Project SamplingBeef Cows – State

• Tier 1 projects are used to verify 

establishment control. Tier 2 projects are exploratory 

in nature. 

• Sampling results are evaluated annually to develop 

the next sampling plan (the Blue Book).

• Results of all residue sampling are used to update 

guidance to PHVs for targeted sampling.

• Residue sampling results 

are published annually in 

the Red Book.

• Sampling results are 

shared with the FDA, and 

used to develop the 

repeat violators list.

• Included in Quarterly 

Establishment Letters

(non-state)

700 482 660

NRP_BV_S National Residue Project Sampling -

Bob Veal – State

NRP_DC_S National Residue Project Sampling -

Dairy Cows – State

NRP_HF_S National Residue Project Sampling -

Heifers – State

NRP_MS_S National Residue Project Sampling -

Market Swine – State

NRP_ST_S National Residue Project Sampling -

Steers – State

NRP_SW_S National Residue Project Sampling -

Sows – State

NRP_YC_S National Residue Project Sampling -

Young Chickens – State

NRP_YT_S National Residue Project Sampling -

Young Turkeys – State

NRP_BC National Residue Project Sampling -

Beef Cows 712 782 712

NRP_BS National Residue Project Sampling -

Bull/Stag 100 171 100

NRP_BV National Residue Project Sampling -

Bob Veal 356 328 356

NRP_DC National Residue Project Sampling -

Dairy Cows 712 833 712

NRP_FFV National Residue Project Sampling -

Formula-Fed Veal 100 158 150
66

Project Code Sampling Project Name Analyses Conducted and Decisions Based 

on Results from Sampling Project Results Reporting

# Samples 

FY18 Sampling 

Plan

# Samples 

Analyzed 

FY18

# Samples FY19 

Sampling Plan

NRP_HC National Residue Project Sampling -

Heavy Calf

NRP_NFFV National Residue Project Sampling -

Non-Formula-Fed Veal

NRP_GO National Residue Project Sampling –

Goats 300 332 300

NRP_HF National Residue Project Sampling –

Heifers 356 369 3,556

NRP_MS National Residue Project Sampling -

Market Swine 712 744 712

NRP_OBT National Residue Project Sampling -

Old Breeder Turkeys 0 1 0

NRP_RS National Residue Sampling Project 

Sampling - Roaster Swine 300 309 300

NRP_SH National Residue Project Sampling –

Sheep 150 168 150

NRP_ST National Residue Project Sampling –

Steers 356 387 356

NRP_SW National Residue Project Sampling -

Sows 712 639 712

NRP_YC National Residue Project Sampling -

Young Chickens 712 696 712

NRP_YT National Residue Project Sampling -

Young Turkeys 712 747 712

PATH_LIVESTK Samples for Histopathological 

Examination - Livestock

• Data are used to inform carcass disposition 

determinations.

• No overall surveillance.

• Included in Quarterly 

Establishment Letters

N/A 3,480 N/A

PATH_OTHER Samples for Histopathological 

Examination - Other (non-meat)

PATH_POULTRY Samples for Histopathological 

Examination - Poultry

PATH_PRODUCT Samples for Histopathological 

Examination – Product
67

Project Code Sampling Project Name Analyses Conducted and Decisions Based 

on Results from Sampling Project Results Reporting

# Samples 

FY18 Sampling 

Plan

# Samples 

Analyzed 

FY18

# Samples FY19 

Sampling Plan

RES_FI Domestic Siluriformes Testing for 

Residue and Speciation

• Data are being analyzed to determine Agency next 

steps.

• Sampling results are evaluated annually to develop 

the next sampling plan (the Blue Book).

• Sampling results are shared with the FDA, and used 

to develop the repeat violators list.

• Results of all residue sampling are used to update 

guidance to PHVs for targeted sampling.

650 636 650

RLMCONT_EL

RLMCONT_MWL; 

RLMCONT_WL

Risk-based Verification - FSL Lab - RTE 

- Food Contact Surface Sample -

Listeria

• Individual results can lead to regulatory action.

• Verification of 9 CFR 430 requirements to control 

Listeria monocytogenes in post-lethality RTE products 

and food contact surfaces. 

• Evaluation of potential cross-contamination or 

harborage of Listeria monocytogenes

• Results of sampling inform PHRE evaluation 

•Evaluation of trends in product positives

• No overall surveillance.

• Included in Quarterly 

Establishment Letters

1,150 3,283 4,218

RLMENVC_EL; 

RLMENVC_MWL; 

RLMENVC_WL

Risk-based Verification - FSL Lab - RTE 

- Non-Food Contact Surface SampleListeria115 331 423

RLMPRODC_EL; 

RLMPRODC_MWL; 

RLMPRODC_WL 

Risk-based Verification- FSL Lab - RTE 

- Listeria - Product Sample 

(Composite)

115 335 423

RTEPROD_RAND RTEPROD Sampling - Random RTE 

Products

• Individual results can lead to regulatory action.

• Verification of 9 CFR 430 requirements to control 

Listeria monocytogenes in post-lethality exposed RTE 

products.

• Evaluation of potential harborage of Listeria 

monocytogenes (if historical isolates exist for 

comparison).

• Verification of lethality treatment effectiveness 

(primarily Salmonella results).

•Results of sampling inform PHRE evaluation

•Evaluation of trends in product positives.

• In response to a continued trend in beginning of FY19 

of under-collection in the RAND project, additional 

sampling tasks have been added for the remainder of 

FY19.

• Aggregate data posted 

on FSIS website

• Included in Quarterly 

Establishment Letters

7,400 7,089 7,400

RTEPROD_RISK RTEPROD Sampling - Risk-based RTE 

Products 7,400 7,974 7,400

SPECID Species Identification Sampling • Conducted ad hoc.

• Individual results can lead to regulatory action.

• No overall surveillance

or reports

N/A 2 N/A
68

Project Code Sampling Project Name Analyses Conducted and Decisions Based 

on Results from Sampling Project Results Reporting

# Samples 

FY18 Sampling 

Plan

# Samples 

Analyzed 

FY18

# Samples FY19 

Sampling Plan

UNKSUB Samples for Identification of 

Unknown Substance

• No overall surveillance

or reports

2
69

Appendix G: Details of Costs Associated with Sampling Projects

Table G1 provides a summary of estimates of costs per sampling project, and includes the discipline for the type of analysis that is typically 

performed for the sampling project. For more detail, please see a description of what is included in Phase 5 (page 38).

Table G1: Estimate of Costs by Sampling Project

Lab Project List PHIS Name

Field Personnel Laboratory Total Cost 

Estimate per 

sample ($)

IPP or 

non-IPP Minutes Costs per sample ($)1

Analysis Type Costs per 

sample ($)

ABNCONT Abnormal Container IPP 54 28.38 Chemical Residue 115.06 143.44

AMR01 Advanced Meat Recovery Product IPP 54 28.38 Pathology 96.41 124.75

AMS_PROD_RTE AMS RTE Canada EV Project - Product Collected by FSIS IPP 54 28.38 Microbiological 82.17 110.55

CG_RES_EL Collector Generated - Residue - Eastern Lab IPP 54 28.38 Chemical Residue 115.06 143.44

CG_RES_MWL Collector Generated - Residue - Midwestern Lab IPP 54 28.38 Chemical Residue 115.06 143.44

CG_RES_WL Collector Generated - Residue - Western Lab IPP 54 28.38 Chemical Residue 115.06 143.44

CG_SHOW_MWL Collector Generated-Residues-Show Animals-Midwest Lab IPP 54 28.38 Chemical Residue 115.06 143.44

CG_SHOW_WL Collector Generated-Residue-Show Animals-Western Lab IPP 54 28.38 Chemical Residue 115.06 143.44

COMPLIAN Investigative Sampling Non-IPP 54 28.38 Microbiological 82.17 110.55

EGGIMP Import-Egg Products-Salmonella and Lm IPP 54 28.38 Microbiological 82.17 110.55

EM31 Egg Product Sampling-Pasteurized-Egg Whites-Salmonella and Lm IPP 54 28.38 Microbiological 82.17 110.55

EM32 Egg Product Sampling-Pasteurized-Whole Egg or Yolks - Salmonella and Lm IPP 54 28.38 Microbiological 82.17 110.55

EM33 Egg Product Sampling-Pasteurized-Whole Eggs with Added Yolks or Whole Egg 

Blends-Salmonella and Lm

IPP 54 28.38 Microbiological 82.17 110.55

EM34 Egg Product Sampling-Pasteurized-Whole Eggs or Yolks with > 2% salt or sugar 

added-Salmonella and Lm

IPP 54 28.38 Microbiological 82.17 110.55

EM35 Egg Product Sampling-Pasteurized-Dried Yellow Egg Products - Salmonella and Lm IPP 54 28.38 Microbiological 82.17 110.55

EM36 Egg Product Sampling-Pasteurized-Dried Egg Whites - Salmonella and Lm IPP 54 28.38 Microbiological 82.17 110.55

EM37 Egg Product Sampling-Pasteurized-Pan Dried Egg Whites - Salmonella and Lm IPP 54 28.38 Microbiological 82.17 110.55

EXP_CH_MSK01 Exploratory Sampling for Mechanically Separated Chicken IPP 54 28.38 Microbiological 82.17 110.55

EXP_CPT_OT01 Exploratory Sampling for Chicken Parts - Others Parts IPP 54 28.38 Microbiological 82.17 110.55

EXP_CPT_QH01 Exploratory Sampling for Chicken Parts - Quarter and Half Carcasses IPP 54 28.38 Microbiological 82.17 110.55

EXP_FI_MIC01 Domestic Siluriformes Sampling for Microbiology IPP 54 28.38 Microbiological 82.17 110.55

EXP_LV_ABX Label Verification for Antibiotics IPP 54 28.38 Chemical Residue 115.06 143.44

EXP_LV_HORM Label Verification for Hormones IPP 54 28.38 Chemical Residue 115.06 143.44

EXP_LV_NUTR Label Verification for Sodium and Fat Content IPP 54 28.38 Chemical Residue 115.06 143.44

EXP_LV_SOY Label Verification for Soy IPP 54 28.38 Pathology 30.38 58.76

EXP_PK_COM02 Baseline Sampling for Pork - Comminuted (Ground, Mechanically Separated, and 

Other Comminuted)

IPP 54 28.38 Microbiological 82.17 110.55

EXP_PK_ICT02 Baseline Sampling for Pork - Intact cuts IPP 54 28.38 Microbiological 82.17 110.55

EXP_PK_NCT02 Baseline Sampling for Pork - Non-Intact Cuts IPP 54 28.38 Microbiological 82.17 110.55

EXP_TU_MSK01 Exploratory Sampling for Mechanically Separated Turkey IPP 54 28.38 Microbiological 82.17 110.55

F_CH_CARC01 Follow-up sampling of Chicken Carcasses IPP 54 28.38 Microbiological 82.17 110.55

F_CH_COM01 Follow-up sampling of Comminuted Chicken IPP 54 28.38 Microbiological 82.17 110.55

F_CPT_LBW01 Follow-up sampling of Chicken Parts; Legs, Breasts, Wings IPP 54 28.38 Microbiological 82.17 110.55
70

Lab Project List PHIS Name

Field Personnel Laboratory Total Cost 

Estimate per 

sample ($)

IPP or 

non-IPP Minutes Costs per sample ($)1

Analysis Type Costs per 

sample ($)

F_TU_COM01 Follow-up sampling of Comminuted Turkey IPP 54 28.38 Microbiological 82.17 110.55

F_TU_CARC01 Follow-up sampling of Turkey Carcasses IPP 54 28.38 Microbiological 82.17 110.55

FAMR01 Follow-up Sampling to Advanced Meat Recovery Positive IPP 54 28.38 Pathology 96.41 124.75

FDS01 Food Defense Sampling - Meals Ready-to-Eat IPP 54 28.38 Microbiological 82.17 110.55

FDS05 Food Defense Sampling- Inauguration Testing IPP 54 28.38 Microbiological 82.17 110.55

FLISTERIA Import-Follow-up to Listeria monocytogenes Positive IPP 54 28.38 Microbiological 82.17 110.55

FOODCHEM Collector-Generated Food Chemistry Samples IPP 54 28.38 Microbiological 82.17 110.55

FRTESALMONEL Import-Follow-up to Salmonella Positive in RTE Product IPP 54 28.38 Microbiological 82.17 110.55

HC_CH_CARC01 HACCP Verification for Young Chicken Carcasses IPP 54 28.38 Microbiological 82.17 110.55

HC_CH_COM01 Sampling for Ground and Other Comminuted Chicken (not Mechanically 

Separated)

IPP 54 28.38 Microbiological 82.17 110.55

HC_CPT_LBW01 Sampling for Chicken Parts – Legs, Breasts, and Wings IPP 54 28.38 Microbiological 82.17 110.55

HC_TU_CARC01 HACCP Verification for Young Turkey Carcasses IPP 54 28.38 Microbiological 82.17 110.55

HC_TU_COM01 Sampling for Ground and Other Comminuted Turkey (not Mechanically Separated) IPP 54 28.38 Microbiological 82.17 110.55

IMP_PORK Imported Raw Pork Product Sampling IPP 54 28.38 Microbiological 82.17 110.55

IMP_POULTRY Import Sampling of Raw and NRTE Poultry Products IPP 54 28.38 Microbiological 82.17 110.55

IMPABNCONT Import-Abnormal Container IPP 54 28.38 Microbiological 82.17 110.55

IMPAMRBEEF Import - AMR Product - Beef IPP 54 28.38 Pathology 96.41 124.75

IMPFISH_CH_E Import Siluriformes Chemistry Sampling - Eastern Laboratory IPP 54 28.38 Chemical Residue 115.06 143.44

IMPFISH_CH_W Import Siluriformes Chemistry Sampling - Western Laboratory IPP 54 28.38 Chemical Residue 115.06 143.44

IMPFISH_MI Import Siluriformes Microbiology Sampling IPP 54 28.38 Microbiological 82.17 110.55

IMPMETALS National Residue Plan - Import Sampling - Metals - Eastern Lab IPP 54 28.38 Chemical Residue 115.06 143.44

IMPPESTICIDE National Residue Plan - Import Sampling - Pesticides - Western Lab IPP 54 28.38 Chemical Residue 115.06 143.44

IMPRESEGG Import Residue Sampling - Egg Products IPP 54 28.38 Chemical Residue 115.06 143.44

IMPRESFR_EL National Residue Plan - Import Sampling - Fresh Product - Eastern Lab IPP 54 28.38 Chemical Residue 115.06 143.44

IMPRESFR_WL National Residue Plan - Import Sampling - Fresh Product - Western Lab IPP 54 28.38 Chemical Residue 115.06 143.44

IMPRESPR_EL National Residue Plan - Import Sampling - Processed Products - Eastern Lab IPP 54 28.38 Chemical Residue 115.06 143.44

IMPRESPR_MWL National Residue Plan - Import Sampling - Processed Product - Midwestern Lab IPP 54 28.38 Chemical Residue 115.06 143.44

IMPSPECIESID Import-Species Identification IPP 54 28.38 Microbiological 82.17 110.55

IMVRTE Import-Micro Pathogen Sampling of RTE Products IPP 54 28.38 Microbiological 82.17 110.55

INTCONT_LM_E For-Cause Sampling-Eastern Lab-RTE Food Contact Surface-Listeria Non-IPP 54 28.38 Microbiological 82.17 110.55

INTCONT_LM_M For-Cause Sampling-Midwest Lab-RTE Food Contact Surface-Listeria Non-IPP 54 28.38 Microbiological 82.17 110.55

INTCONT_LM_W For-Cause Sampling-Western Lab-RTE Food Contact Surface-Listeria Non-IPP 54 28.38 Microbiological 82.17 110.55

INTCONT_SA_E For-Cause Sampling-Eastern Lab-RTE Food Contact Surface-Salmonella Non-IPP 54 28.38 Microbiological 82.17 110.55

INTCONT_SA_M For-Cause Sampling-Midwest Lab-RTE Food Contact Surface-Salmonella Non-IPP 54 28.38 Microbiological 82.17 110.55

INTENV_LM_E For-Cause Sampling-Eastern Lab-Non-Food Surface-Listeria Non-IPP 54 28.38 Microbiological 82.17 110.55

INTENV_LM_M For-Cause Sampling-Midwest Lab-Non-Food Surface-Listeria Non-IPP 54 28.38 Microbiological 82.17 110.55

INTENV_LM_W For-Cause Sampling-Western Lab-Non-Food Surface-Listeria Non-IPP 54 28.38 Microbiological 82.17 110.55

INTENV_SA_E For-Cause Sampling-Eastern Lab-Non-Food Surface-Salmonella Non-IPP 54 28.38 Microbiological 82.17 110.55

INTENV_SA_M For-Cause Sampling-Midwest Lab-Non-Food Surface-Salmonella Non-IPP 54 28.38 Microbiological 82.17 110.55

INTPROD_LM_E For-Cause Sampling-Eastern Lab-RTE Product-Listeria IPP 54 28.38 Microbiological 82.17 110.55

INTPROD_LM_M For-Cause Sampling-Midwest Lab-RTE Product-Listeria IPP 54 28.38 Microbiological 82.17 110.55

INTPROD_LM_W For-Cause Sampling-Western Lab-RTE Product-Listeria IPP 54 28.38 Microbiological 82.17 110.55
71

Lab Project List PHIS Name

Field Personnel Laboratory Total Cost 

Estimate per 

sample ($)

IPP or 

non-IPP Minutes Costs per sample ($)1

Analysis Type Costs per 

sample ($)

INTPROD_SA_E For-Cause Sampling-Eastern Lab-RTE Product-Salmonella IPP 54 28.38 Microbiological 82.17 110.55

INTPROD_SA_M For-Cause Sampling-Midwest Lab-RTE Product-Salmonella IPP 54 28.38 Microbiological 82.17 110.55

KIS KIS - Samples from In-plant Testing IPP 54 28.38 Chemical Residue 115.06 143.44

LO_CH_CARC01 Very Low Volume Sampling for Chicken Carcasses IPP 54 28.38 Microbiological 82.17 110.55

LO_CH_COM01 Very Low Volume Sampling for Ground and Other Comminuted Chicken (not 

Mechanically Separated)

IPP 54 28.38 Microbiological 82.17 110.55

LO_CH_MSK01 Very Low Volume Sampling for Mechanically Separated Chicken IPP 54 28.38 Microbiological 82.17 110.55

LO_CPT_LBW01 Very Low Volume Sampling for Raw Chicken Parts - Legs, Breast, Wings IPP 54 28.38 Microbiological 82.17 110.55

LO_CPT_OT01 Very Low Volume Sampling for Raw Chicken Parts - Other Parts IPP 54 28.38 Microbiological 82.17 110.55

LO_CPT_QH01 Very Low Volume Sampling for Chicken Parts - Quarters/Halves IPP 54 28.38 Microbiological 82.17 110.55

LO_TU_CARC01 Very Low Volume Sampling for Turkey Carcasses IPP 54 28.38 Microbiological 82.17 110.55

LO_TU_COM01 Very Low Volume Sampling for Ground and Other Comminuted Turkey (not 

Mechanically Separated)

IPP 54 28.38 Microbiological 82.17 110.55

LO_TU_MSK01 Very Low Volume Sampling for Mechanically Separated Turkey IPP 54 28.38 Microbiological 82.17 110.55

MT05 Raw Ground Beef or Veal Sampling - Retail IPP 54 28.38 Microbiological 82.17 110.55

MT06 Raw Ground Beef or Veal Sampling - Retail - Follow-up IPP 54 28.38 Microbiological 82.17 110.55

MT08 Import-Sampling of Raw Ground or Comminuted Beef or Veal Product IPP 54 28.38 Microbiological 82.17 110.55

MT43 Risk-based Sampling of Raw Ground Beef or Veal Products - E.coli O157:H7 & 

Salmonella

IPP 54 28.38 Microbiological 82.17 110.55

MT44 Follow-up Risk-based Sampling of Positive Raw Ground Beef or Veal Sample -E.coli 

O157:H7/Salmonella

IPP 54 28.38 Microbiological 82.17 110.55

MT44T Follow-up E.coli Sampling of Raw Ground Beef, Trimmings or Components 

(Traceback)

IPP 54 28.38 Microbiological 82.17 110.55

MT51 Import-Raw Beef Manufactured Trimmings or Components for use in Ground Beef 

or Beef Products

IPP 54 28.38 Microbiological 82.17 110.55

MT52 Follow-up Sampling of Suppliers of Raw Ground Beef Trim or Components IPP 54 28.38 Microbiological 82.17 110.55

MT53 Follow-up Sampling of Beef Mfg. Trim or Other Raw Ground Beef or Beef Patty 

Components

IPP 54 28.38 Microbiological 82.17 110.55

MT60 Sampling of Beef Manufacturing Trimmings2IPP 108 56.75 Microbiological 82.17 138.92

MT64 Sampling of Raw Ground Beef or Beef Patty Components (other than trim) IPP 54 28.38 Microbiological 82.17 110.55

MT65 Sampling of Bench Trim for further use in ANY raw, non-intact beef products2IPP 108 56.75 Microbiological 82.17 138.92

NARMS_BC NARMS-National Antimicrobial Resistance Monitoring System Sampling-Beef Cows IPP 54 28.38 Microbiological 82.17 110.55

NARMS_DC NARMS-Dairy Cows (Paired with RESNARMS_DC) IPP 54 28.38 Microbiological 82.17 110.55

NARMS_HF NARMS-National Antimicrobial Resistance Monitoring System Sampling-Heifers IPP 54 28.38 Microbiological 82.17 110.55

NARMS_MS NARMS-National Antimicrobial Resistance Monitoring System Sampling-Market 

Swine

IPP 54 28.38 Microbiological 82.17 110.55

NARMS_ST NARMS-National Antimicrobial Resistance Monitoring System Sampling-Steers IPP 54 28.38 Microbiological 82.17 110.55

NARMS_SW NARMS-National Antimicrobial Resistance Monitoring System Sampling-Sows IPP 54 28.38 Microbiological 82.17 110.55

NARMS_YC NARMS-Young Chickens (Paired with RESNARMS_YC) IPP 54 28.38 Microbiological 82.17 110.55

NARMS_YT NARMS-National Antimicrobial Resistance Monitoring System Sampling-Young 

Turkeys

IPP 54 28.38 Microbiological 82.17 110.55

NRP_BC National Residue Project Sampling - Beef Cows IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_BC_S National Residue Project - Beef Cows - State IPP 54 28.38 Chemical Residue 115.06 143.44
72

Lab Project List PHIS Name

Field Personnel Laboratory Total Cost 

Estimate per 

sample ($)

IPP or 

non-IPP Minutes Costs per sample ($)1

Analysis Type Costs per 

sample ($)

NRP_BS National Residue Project Sampling - Bull/Stag IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_BV National Residue Project Sampling - Bob Veal IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_BV_S National Residue Project - Bob Veal - State IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_DC National Residue Project Sampling - Dairy Cows IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_DC_S National Residue Project - Dairy Cows - State IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_FFV National Residue Project Sampling - Formula-Fed Veal IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_FS National Residue Project Sampling – Feral Swine IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_GO National Residue Project Sampling - Goats IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_HC National Residue Project Sampling - Heavy Calf IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_HF National Residue Project Sampling - Heifers IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_HF_S National Residue Project - Heifers - State IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_MS National Residue Project Sampling - Market Swine IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_MS_S National Residue Project - Market Swine - State IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_NFFV National Residue Project Sampling - Non Formula-Fed Veal IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_OBT National Residue Project - Old Breeder Turkeys IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_RS National Residue Sampling Project - Roaster Swine IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_SH National Residue Project Sampling - Sheep IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_ST National Residue Project Sampling - Steers IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_ST_S National Residue Project - Steers - State IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_SW National Residue Project Sampling - Sows IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_SW_S National Residue Project - Sows - State IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_YC National Residue Project Sampling - Young Chickens IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_YC_S National Residue Project - Young Chickens - State IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_YT National Residue Project Sampling - Young Turkeys IPP 54 28.38 Chemical Residue 115.06 143.44

NRP_YT_S National Residue Project - Young Turkeys - State IPP 54 28.38 Chemical Residue 115.06 143.44

PATH_LIVESTK Samples for Histopathological Examination - Livestock IPP 54 28.38 Pathology 30.38 58.76

PATH_OTHER Samples for Histopathological Examination - Other (non-meat) IPP 54 28.38 Pathology 30.38 58.76

PATH_POULTRY Samples for Histopathological Examination - Poultry IPP 54 28.38 Pathology 30.38 58.76

PATH_PRODUCT Samples for Histopathological Examination - Product IPP 54 28.38 Pathology 30.38 58.76

RE_CH_CARC01 Religious Exempt Sampling for Chicken Carcasses IPP 54 28.38 Microbiological 82.17 110.55

RES_FI Domestic Siluriformes Testing for Residue and Speciation IPP 54 28.38 Chemical Residue 115.06 143.44

RLMCONT_EL Risk-based Verification-Eastern Lab - RTE - Food Contact Surface Sample - Listeria IPP 54 28.38 Microbiological 82.17 110.55

RLMCONT_MWL Risk-based Verification-Midwest Lab-RTE-Food Contact Surface Sample-Listeria IPP 54 28.38 Microbiological 82.17 110.55

RLMCONT_WL Risk-based Verification-Western Lab-Food Contact Surface Sample-Listeria IPP 54 28.38 Microbiological 82.17 110.55

RLMENVC_EL Risk-based Verification-Eastern Lab - RTE - Non-Food Contact Surface SampleListeriaIPP 54 28.38 Microbiological 82.17 110.55

RLMENVC_MWL Risk-based Verification-Midwest Lab-RTE-Non-Food Contact Surface SampleListeriaIPP 54 28.38 Microbiological 82.17 110.55

RLMENVC_WL Risk-based Verification-Western Lab-RTE-Non-food Contact Surface SampleListeriaIPP 54 28.38 Microbiological 82.17 110.55

RLMENVR_EL Risk-based Verification-Eastern Lab-RTE-Non-food Contact Brine Sample IPP 54 28.38 Microbiological 82.17 110.55

RLMENVR_MWL Risk-based Verification-Midwest Lab-RTE-Non-food Contact Brine Sample IPP 54 28.38 Microbiological 82.17 110.55

RLMENVR_WL Risk-based Verification-Western Lab-RTE-Non-food Contact Brine Sample IPP 54 28.38 Microbiological 82.17 110.55
73

Lab Project List PHIS Name

Field Personnel Laboratory Total Cost 

Estimate per 

sample ($)

IPP or 

non-IPP Minutes Costs per sample ($)1

Analysis Type Costs per 

sample ($)

RLMPRODC_EL Risk-based Verification-Eastern Lab - RTE - Listeria - Product Sample (Composite) IPP 54 28.38 Microbiological 82.17 110.55

RLMPRODC_MWL Risk-based Verification-Midwestern Lab - RTE - Listeria - Product Sample 

(Composite)

IPP 54 28.38 Microbiological 82.17 110.55

RLMPRODC_WL Risk-based Verification-Western Lab - RTE - Listeria - Product Sample (Composite) IPP 54 28.38 Microbiological 82.17 110.55

RTEPROD_RAND RTEPROD Sampling - Random RTE Products IPP 54 28.38 Microbiological 82.17 110.55

RTEPROD_RISK RTEPROD Sampling - Risk-based RTE Products IPP 54 28.38 Microbiological 82.17 110.55

SPECID Species Identification Sampling IPP 54 28.38 Microbiological 82.17 110.55

UNKSUB Samples for Identification of Unknown Substance IPP 54 28.38 Microbiological 82.17 110.55

1 The field hourly rate is $31.53. Cost per sample derived by multiplying the hourly rate by the fractional hours. For instance 54 minutes equates to 0.9 hours. This is then multiplied by the hourly rate to get the field cost 

per sample.

2 N-60 Composite Sample
74

Appendix H: Details by Project of Master Table

Table H1 provides a graphical representation while H2 gives the tabular representation of the benefit score and 

categorized cost per sample for each individual sampling project, with individual projects sorted by group. The costs 

have been categorized into low, medium and high in order to better compare cost/benefit between projects. This is a 

similar presentation to Table ES1, which presented this information summarized by sampling group. 

Table H1: Detailed Master Table Summary by Individual Project Scatter Plot
75

Table H2: Detailed Master Table Summary by Individual Project Tablix