EVALUATION OF U.S. EPA’S SCREENER QUESTIONNAIRE FOR 

THE AQUATIC ANIMAL PRODUCTION INDUSTRY AND 

COMPARISON TO USDA DATA

Marla D. Smith and Marta E. Jordan

United States Environmental Protection Agency 

1200 Pennsylvania Ave., NW (4303T), Washington, DC 20460; smith.marla@epa.gov and jordan.marta@epa.gov

Abstract

In August 2002, the U.S. Environmental Protection Agency (EPA) proposed national regulations to control 

pollutant discharges from the wastewaters of the aquatic animal production (AAP) industry. In developing this 

proposed rule, EPA collected information from the AAP industry by using a two-phase sample design. In the 

first phase, EPA conducted a census of the industry using a screener questionnaire that requested basic 

information to better identify and define the population. In the second phase, EPA used a second questionnaire 

to collect detailed site-specific technical and financial information from a sample of the screener questionnaire 

respondents. After adjusting for nonresponse in the screener questionnaire responses, EPA estimated that 3,075 

facilities conducted AAP in 2000 compared to 4,028 facilities identified by the 1998 Census of Aquaculture 

conducted by the U.S. Department of Agriculture (USDA). This paper evaluates EPA’s screener questionnaire 

design and compares its results to the USDA Census of Aquaculture and other data sources from the USDA. 

Introduction

This paper describes data that the U.S. Environmental Protection Agency (EPA) is using to develop national regulations to control 

pollutant discharges from the wastewaters of the aquatic animal production (AAP) industry. In developing these regulations, EPA 

used a two-phase survey to collect information from the AAP industry. In the first phase, EPA used a census to collect basic 

information that would allow EPA to better identify and define the population. In the second phase, EPA collected detailed sitespecific technical and financial information from a sample of the first-phase respondents. After adjusting for nonresponse in the 

screener questionnaire responses, EPA estimated that 3,075 facilities conducted AAP in 2000 compared to 4,028 facilities 

identified by the U.S. Department of Agriculture (USDA) 1998 Census of Aquaculture. This paper evaluates EPA’s screener 

questionnaire in the first phase of the survey and compares those results to data sources from the USDA. 

Background and Definition

During the last 30 years, EPA has conducted surveys of numerous industrial sectors. The information from these surveys has been 

used to develop national regulations for water pollution control. For each industry, EPA selects an appropriate technology control, 

by determining its availability within the industry, the potential costs to the industry, the economic impacts of the costs on 

businesses, as well as the degree to which it effectively prevents the generation of pollutants, or removes, controls, or reduces 

specific pollutants. EPA then applies the technology control to determine appropriate numerical limits on the amount of specific 

pollutants that can be discharged in wastewater by facilities in each industry. Since 1974, this program has been very successful, 

resulting in a combined reduction of over 690 billion pounds of pollutants each year to the nation’s waters due to regulations on 

more than 55 industrial categories. 

Although some AAP facilities currently have permits to discharge their wastewater, EPA has not promulgated national regulations 

for this particular industry. The AAP industry includes fish hatcheries and farms that grow finfish (such as trout, catfish, salmon, 

tilapia, hybrid striped bass, ornamentals), shrimp, and other aquatic animals. Some AAP facilities can contribute nutrients to

environmentally sensitive areas in estuaries, rivers, lakes, and streams throughout the country. In addition, the facilities use and 

potentially discharge drugs (e.g., antibiotics) and other chemicals. During its examination of this industry, EPA has observed

improvements in wastewater treatment within the AAP industry by some facilities to reduce nutrients and other pollutants. By 

examining the cost and performance of pollution control technologies and practices, EPA is investigating whether more facilities 

could employ these technologies to reduce pollutant discharges to surface waters. 

Data Collection

Page 1
To support its evaluation of the AAP industry, EPA has gathered, and continues to gather, technical and economic data to define

the industry, develop the industry profile, develop technology options, estimate the environmental benefits of regulation, evaluate 

economic achievability of the regulatory options, and determine practices leading to pollutant reductions. The major sources of 

information are EPA’s two-phase survey, the USDA 1998 Census of Aquaculture, EPA visits to AAP facilities, literature searches, 

and information provided by industry experts (including a Task Force chaired by the USDA and the National Oceanic and 

Atmospheric Administration (NOAA)). This paper focuses on the data from the first phase of EPA’s survey and the USDA 

Census. 

EPA: Screener Questionnaire for the Aquatic Animal Production Industry

In August 2001, EPA conducted the first-phase of its two-phase survey of the AAP industry. In this first phase, EPA mailed the 

Screener Questionnaire for the Aquatic Animal Production Industry (“screener”) to approximately 6000 facilities. Response was 

mandatory under Section 308 of the Clean Water Act. Because the questionnaire required only one hour to complete, EPA 

requested that responses be returned within 15 days. The questionnaire consisted of 11 questions about the type of facility (e.g., 

commercial, government, tribal), the quantities of animals produced by species and size category, the production methods that 

were used, water discharges from the property, and description of pollutant control practices. 

USDA: 1998 Census of Aquaculture

The USDA notes that the 1998 Census of Aquaculture (“Census”) was the first census to take a comprehensive snapshot of all 

species produced throughout the 50 states.1

 This survey was conducted by USDA’s National Agricultural Statistics Service 

(NASS) to respond to “the intense need for an accurate measurement of the aquaculture sector, which grew from $45 million for 

value of products sold in 1974 to over $978 million in the 1998.”2

 The USDA Census collected information about on-farm 

aquaculture practices in 1998 including the size of the operation based on water area, production, sales, sources of water, point of 

first sale outlets, cooperative agreements and contracts, and aquaculture distributed for restoration or conservation practices. 

Information Collected by Both Surveys

While the USDA Census collected more information than the EPA screener, both agencies obtained information on production 

methods, species produced, and production levels. 

Comparisons

Because it relied on both data sources for its analyses, EPA compared the EPA screener estimates to the USDA Census counts. 

Because the population sizes appeared to differ without obvious explanations in some cases, EPA investigated the differences 

between the two data sources. The following sections describe the population estimates, the definition of the target population, the 

frame, and base year. 

Population Estimates

From the screener responses, EPA estimated 3075 facilities conducted AAP. This estimate was derived from adjusting the 2329 

in-scope respondents for nonresponse on a state-by-state basis. (EPA determined that using a state-by-state adjustment was 

appropriate because the type of aquaculture and production practices would tend to be similar within a state.) In contrast, the 

USDA counted 4028 farms. The USDA had an extensive follow-up campaign for its 1998 Census, and, as a result, assumed that 

all farms responded to the Census. The two sources also differed in the number of facilities growing each species and using 

different production methods. 

Species. In terms of the number of facilities growing particular species, the ratio of the USDA’s count to EPA’s estimate varied 

from 0.17 to 6.78, as shown in Table 1. In addition, the 95 percent confidence intervals for the EPA estimates do not overlap with 

the USDA counts for any of the species. 

 

1U.S. Department of Agriculture (2000), 1997 Census of Agriculture: Census of Aquaculture (1998), Volume 3, Special 

Studies, Part 3, AC97-SP-3, page v. 

2

Ibid.

Table 1. Number of Facilities Growing Species 

Page 2
Species USDA Census EPA Screener 

(count) 

 

Estimated 

Number 

Standard 

Error 

Ratio 

(USDA/EPA) 

Trout 561 818 28 0.69 

 

Catfish 1370 901 30 1.52 

Salmon 47 277 16 0.17 

 

 

 

 

 

Tilapia 148 178 15 0.83 

Crawfish 563 83 9 6.78 

Production Method. The two data sources again displayed different values for the production methods, with ratios ranging from 

0.27 to 1.55, as shown in Table 2. In addition, the 95 percent confidence intervals for the EPA estimates do not overlap with the 

USDA counts for any of the production methods. 

Table 2. Number of Facilities Using Production Method 

Production Method USDA Census EPA Screener 

(count) 

Estimated 

Number 

Standard 

Error 

Ratio 

(USDA/EPA) 

Ponds 2878 

 

1860 43 1.55 

Flow through systems 617 1358 36 

 

0.45 

Net pens and cages 167 610 26 0.27 

Recirculating systems 328 262 17 1.25 

Other 569 403 not calc. 1.41 

Definition of Target Population

Because EPA and USDA defined the target populations differently, EPA investigated the impact of those definitions on the values. 

 EPA defined the AAP industry to include all “facilities” that answered “Yes” to the question “Do you produce (grow) aquatic 

animals (fish, shellfish, or other aquatic animals) at this facility?” The USDA defined the target population as “any commercial or 

noncommercial place from which $1,000 or more of aquaculture products were sold or normally would have been sold during the 

census year.” Thus, one difference between the two target populations was that the USDA required that facilities sell their 

products, while EPA did not. As a result of this exclusion, the USDA generally excluded facilities such as state hatcheries. 

Consequently, EPA would have expected its total population estimate to exceed the USDA count, which was not the case. 

However, for certain species typically grown in state hatcheries (such as trout and salmon), EPA’s estimates exceeded the USDA 

counts. Another difference in the target population definitions was that the USDA included 20 farms3

 that sold “algae and sea 

vegetables”, which EPA excluded because its target population focused on aquatic animals only. 

Frame

 

3

Ibid., Table 19. 

Page 3
Because the definitions of the target population lead to different expectations than observed, EPA then evaluated the sources used 

to develop the frame. The USDA’s frame for the 1998 Census consisted of farms that identified themselves as having “fish and 

other aquaculture products” in the 1997 Census of Agriculture. This frame was supplemented by other USDA information, and 

lists of state and federal fish hatcheries. Because of confidentiality restrictions, the USDA was unable to share its frame with 

EPA. As a consequence, at the start of its regulatory activities in 2000, EPA relied primarily on the Dun & Bradstreet (D&B) 

database. From this database, EPA selected all facilities with Standard Industrial Classification (SIC) codes for animal aquaculture 

(0273), animal specialties (0279), and fish hatcheries and preserves (0921). In addition, EPA requested lists from states, tribes, 

and various associations. EPA also consulted aquaculture journals and conducted searches through the internet. These requests

were only partly successful because some state agencies were unable to share their lists of aquaculture facilities with EPA. 

Finally, as a result of mailing the questionnaire to facilities, they, in turn, identified other facilities that did not receive the 

questionnaire. As a result of consolidating the lists and eliminating the obvious duplicates, EPA’s frame consisted of 

approximately 6000 facilities. However, of the 4214 returned questionnaires, only 2329 were in-scope (i.e., facilities that 

conducted AAP). Because the USDA frame would have excluded farms with less than $1,000 of agricultural products produced 

and sold, EPA expected that the EPA frame would be more comprehensive. It is possible that EPA’s other sources of information 

did not identify all of the facilities in states that were unable to provide EPA with information about AAP facilities.4

 Perhaps, the 

USDA, a non-regulatory agency, was able to compile a frame that includes these facilities. 

 

 

 

 

Base year

EPA also considered whether the AAP population changed from 1998, the USDA Census base year, to 2000, the EPA base year. 

For this evaluation, EPA used two additional USDA sources. The first additional source was the USDA Census of Agriculture

which provides data for 1997, and was used to develop the USDA’s frame for the 1998 Census of Aquaculture. The second 

additional source was the USDA report Catfish and Trout Production (“report”) which provides data for 2000, which is the same 

as EPA’s base year. (The USDA developed this report through monthly catfish processing, quarterly catfish production, and 

annual catfish and trout surveys.) 

Trout. For trout production, the numbers of facilities increased over time in the three USDA data sources, but even the most 

recent USDA value of 690 facilities was less than the EPA screener estimate of 818 facilities. EPA’s larger estimate may include 

state hatcheries and others that were not part of the USDA target population for its Censuses. 

Catfish. For catfish production, the three USDA data sources provided population values that decreased over time from 1457 to 

1252. The EPA estimate of 901 facilities growing catfish is lower than any of these USDA values. Unlike trout which is 

sometimes grown by noncommercial facilities such as state hatcheries, EPA has found only commercial growers of catfish, and 

thus, EPA had expected its estimate and the USDA values to be similar because the target populations would be the same. Table 3 

summarizes the number of facilities growing trout and catfish according to the three USDA sources and the EPA screener. The 

table also provides the standard error of the EPA estimates. 

Table 3. Number of Facilities Producing Trout and Catfish in 2000 

Species USDA Agriculture EPA Screener (2000) 

Census (1997) 

USDA Aquaculture 

Census (1998) 

USDA Report 

(2000) 

Estimate Std. error 

Trout 411 

 

561 690 818 28 

Catfish 1457 1370 1252 901 30 

Assessment of EPA Screener

In addition to evaluating the differences between the surveys, EPA also assessed its screener by examining the response rate and 

the scoping question. 

Response Rate 

 

4However, for one state, EPA and USDA values were similar. (EPA has only reviewed the data for this one state at this 

time.) 

Page 4
The response rate was 76 percent for the screener, which is lower than similar surveys conducted by this program. As is its usual 

practice, after mailing the screener, EPA conducted a follow-up campaign that included reminder letters and telephone calls (when 

the phone numbers were readily available). Even with its follow-up campaign, EPA believes that the response rate for the AAP 

industry was affected by the large out-of-scope component in its frame, with some out-of-scope facilities unaware of the 

importance of returning the survey. EPA also encountered some resistance to the requirement to respond to the screener. While

some facilities contacted EPA to express their concerns and their refusals, EPA expects that other facilities merely ignored the 

requirement. EPA expects that the USDA would encounter less resistance to its surveys, because the USDA is not primarily a 

regulatory agency. 

Scoping Question

EPA also noted that some respondents expressed some confusion about the term “produced” that was used in the scoping question 

“Do you produce (grow) aquatic animals (fish, shellfish, or other aquatic animals) at this facility?” For example, some 

respondents questioned whether “produced” meant “sales.” Also, respondents for aquariums did not always consider that the 

facility “produced” or “grew” any aquatic animals because they were not breeding the animals in any concentrated manner. As a 

result of this confusion, some respondents considered themselves to be out-of-scope, when EPA later identified them as in-scope

facilities. 

In addition, EPA is concerned that facilities may have been confused about the time period covered by the screener, because the

scoping question did not explicitly identify the base year. Thus, some facilities may have assumed that the screener was asking 

about conditions when they received the screener rather than the conditions during any period in 2000. As a result, facilities with 

seasonal production may have considered themselves to be out-of-scope. 

Conclusions

EPA did not find any factors that would fully explain the differences between the USDA counts and EPA estimates. Although the 

EPA estimated total population size was lower, the EPA target population was more inclusive than the USDA Census. Despite its 

incorporation of mailing addresses from many sources, some facilities were apparently still missing from EPA’s frame. While the 

base year was different for the USDA Census and the EPA screener, EPA examined other USDA data sources and found that the 

USDA values were more similar to each other than the EPA screener estimated values. In examining the response rate, EPA noted 

that some facilities refused to respond to the questionnaire and some out-of-scope facilities may have failed to realize the 

importance of responding. EPA also noted some confusion about the scoping question that lead some facilities to conclude that 

their operations were out-of-scope for the screener. Because these factors do not appear to fully explain the differences, EPA

concludes that there are still other possible explanatory factors. Partly as a consequence of these discrepancies, EPA does not 

intend to rely solely on the USDA data or EPA survey data to develop its regulations for the AAP industry. EPA is supplementing 

these data with other sources such as literature searches, visits to AAP facilities, and industry experts (such as the Joint 

Subcommittee on Aquaculture’s Aquaculture Effluents Task Force (JSA/AETF) which includes experts from various government 

agencies, industry and trade associations, academia, and other interested stakeholders). In addition, the preamble to the proposed 

rule (August 2002) solicited information, comment, and data about a number of aspects considered in development of the proposed

regulations. 

References

Office of the Federal Register, Federal Register, Volume 67, Number 177, pages 57872-57928 (located at 

http://www.epa.gov/fedrgstr/EPA-WATER/2002/September/Day-12/w21673.pdf). 

U.S. Department of Agriculture (1998), 1998 Census of Aquaculture, OMB Control No. 0535-0237 (located at 

http://www.nass.usda.gov/census/census97/aquaculture/reportform.pdf). 

U.S. Department of Agriculture (1999), 1997 Census of Agriculture: United States: Summary and State Data, Volume 

1,Geographic Area Series, Part 51, AC97-A-51, Table 23 (pages 425-6) and Appendix D (Table 23 located at 

http://www.nass.usda.gov/census/census97/volume1/us-51/us2_21.pdf). 

U.S. Department of Agriculture (2000), 1997 Census of Agriculture: Census of Aquaculture (1998), Volume 3, Special Studies, 

Part 3, AC97-SP-3 (located at http://www.nass.usda.gov/census/census97/aquaculture/aquaculture.htm). 

Page 5
U.S. Department of Agriculture (2000), “Catfish and Trout Production,” Aq 2 (2-01) (located at 

http://usda.mannlib.cornell.edu/reports/nassr/other/ztp-bb/cfpd0201.pdf). 

U.S. Environmental Protection Agency (2000), “Fact Sheet: EPA Expands Focus on Nutrient Pollution,” EPA-821-F-00-002 

(located at http://epa.gov/waterscience/guide/aquaculture/factsheet.html). 

U.S. Environmental Protection Agency (2001), Screener Questionnaire for the Aquatic Animal Production Industry, OMB 

Control No. 2040-0237 (located at http://epa.gov/waterscience/guide/aquaculture/screenersurvey.pdf). 

U.S. Environmental Protection Agency (2002a), Development Document for Proposed Effluent Limitations Guidelines and 

Standards for the Concentrated Aquatic Animal Production Industry Point Source Category, EPA-821-R-02-016 (located 

at http://epa.gov/guide/aquaculture/tdd/tdd.html), Chapter 3 and Appendix A. 

U.S. Environmental Protection Agency (2002b), “Fact Sheet: Proposed Effluent Guidelines for Aquatic Animal Production,” EPA 

821-F-02-016 (located at http://epa.gov/guide/aquaculture/factsh2.pdf). 

Westat (2002a). “Alternative weighting plan.” DCN 50050 in Section 10.3 of EPA Docket W-02-01. 

Westat (2002b). “Comments on screener questionnaire design.” DCN 50064 in Section 10.3 of EPA Docket W-02-01. 

Page 6