1 

Closing the Feedback Loop in Survey Design – How Public Sector 

Research Needs Inform Evolution of the Census of Agriculture 

Luanne Lohr1, Donald Buysse2 

1US Department of Agriculture Agricultural Marketing Service, Luanne.Lohr@ams.usda.gov, 2US Department of 

Agriculture National Agricultural Statistics Service, Donald.Buysse@nass.usda.gov 

Introduction 

The Census of Agriculture is an enumerative and descriptive survey of American farm and ranch operators that is 

conducted every five years by the U.S. Department of Agriculture’s National Agricultural Statistics Service (USDANASS). For Census purposes, a farm is any place from which $1,000 or more of agricultural products were 

produced and sold, or normally would have been sold, during the Census year. Response is mandated by law. Data 

are collected on land use and ownership, operator characteristics, demographics, production practices, income, 

expenditures, and other areas. 

The Census of Agriculture provides the only source of uniform, comprehensive agricultural data that is tabulated for 

every county in the nation. Census data are used by public sector researchers to discover trends, predict program 

outcomes, evaluate program effectiveness, and respond to legislative requests for information prior to policy 

formulation. Two goals for data development are stability – obtaining the same statistical data over the years to 

establish a time series – and responsiveness – including questions to elicit information on emerging topics that are of 

interest to consumers of the data products. 

To address these goals, the USDA-NASS may add or reformulate questions in the Census to identify key subgroups, 

from which additional information may be collected in Special Studies, referred to as census follow-on surveys. 

There were four follow-on surveys conducted after the last census of agriculture in 2007. Among the four were the 

2008 Organic Production Survey and 2009 On-Farm Renewable Energy Production Survey. The organic and 

renewable energy follow-ons were new toUSDA-NASS and were developed in response to an increased need for 

more detailed information regarding these key subgroups. In each instance, the survey population was established 

based on specific responses to questions in the 2007 Census of Agriculture. 

Given the expense of designing, testing, and conducting the Ag Census, changes require strong justification. This 

paper explores the feedback loop between public sector researchers and the design of the Census of Agriculture. We 

describe the process of design evolution resulting from policy and research needs, and offer two examples of 

feedback loops that resulted in changes to the Census questionnaire to initiate data gathering on organic production 

and local foods marketing. We discuss the challenges of responding to researcher and policy maker needs. 

Census of Agriculture Data Usage

The value of conducting a national census of agricultural production has long been recognized. The first agricultural 

census was taken in 1840 as part of the decennial population census. Up until 1996, the U.S. Department of 

Commerce was responsible for collecting agriculture data. The 1997 Appropriations Act transferred the 

responsibility to the USDA-NASS. The last iteration was the 2007 Census of Agriculture, which marked the 27th

Federal agriculture census but only the third executed by USDA-NASS. The next census of agriculture is scheduled 

for reference year 2012. 

In a review of the Census of Agriculture conducted at the request of USDA-NASS, The Council on Food, 

Agricultural, and Resource Economics (C-FARE) (2007) commented that the report is one of the most widely used 

sources of data describing the nation’s agriculture sector. The most important challenges noted by C-FARE (2007) 

is dealing with a wide range of often competing uses and demands for the data, the ever changing nature of the 

agriculture sector, and resource constraints in funding the data collection effort. 
2 

 

Information from the census of agriculture is routinely requested by farm organizations, businesses, governmental 

bodies, college and university researchers, and the media outlets. Some uses of agriculture census data include: 

 Evaluate, change, promote, and formulate farm and rural policies and programs that help agricultural 

producers; 

 Study historical trends, assess current conditions, and plan for the future; 

 Formulate market strategies, provide more efficient production and distribution systems, and locate 

facilities for agricultural communities; 

 Allocate local and national funds for farm-related programs, including extension service projects, 

agricultural research, soil conservation programs, and land-grant colleges and universities; 

 Analyze and report on the current state of food, fuel, feed, and fiber production in the United States. 

In providing a justification for conducting a national census, as opposed to a series of less expensive sample surveys, 

C-FARE (2007) pointed out that a census is needed to provide a benchmark for annual estimates of crop acreages 

and livestock inventories on farms, to provide detail for small geographic areas and small crop or livestock segments 

that might not otherwise be large enough to be sampled in a survey, to provide a list frame for inter-census surveys, 

and to allow for cross-tabulations of detailed subpopulations, such as minority-operated farms producing a specific 

crop. The U.S. Department of Agriculture-Economic Research Service (USDA-ERS) is one such beneficiary, using 

the Census of Agriculture results to benchmark results in its annual Agricultural Resource Management Survey 

(ARMS). 

Content Development 

Consistent with the previous two agriculture censuses, the 2012 report form contains 24 pages of questions related to 

the nation’s agricultural activity. Since one of the primary objectives of the census of agriculture is to provide a 

count of all farms and ranches, much of the content is made up of questions related to agriculture production used to 

determine whether an operation is a farm. Most of the remaining content provides valuable data about operator 

characteristics, production practices, expenses, and farm related income and activities. 

C-FARE (2007) urged USDA-NASS to maintain focus on agriculture as an economic sector, but to respond to 

stakeholder requests for more data on marketing, production, environmental practices, government payments, and 

farm labor. USDA-NASS was also reminded to account for the diversity in scale and purpose of farm operations, 

from large complex agribusinesses to “lifestyle” farmers. 

After including the main content items, USDA-NASS is left with very little space in the form to address these 

additional concerns. In order to increase the relevance of the Census Programs, beginning with the 2007 Census of 

Agriculture, USDA-NASS reserved some space for emerging issues. To limit the area these emerging issues account 

for, a simple yes/no screening question is asked of all respondents. These screening questions are concentrated in the 

“Practices Section” of the census report form. One use for responses to these questions is to develop sampling 

frames for subsequent detailed follow-on surveys of populations who engage in a particular practice. C-FARE 

(2007) recommended that USDA-NASS continue to use follow-on surveys as a least-cost means of collecting data 

items that do not apply for all farms or all counties, and consult with other federal agencies and stakeholders as to 

what follow-on surveys would be most relevant. 

Even before publishing the results of the census of agriculture, work begins on developing the next iteration of the 

questionnaire. USDA-NASS solicits content items during its data user meetings and also publishes a Federal 

Register Notice asking for input on new content items. All items received are considered but only a limited number 

are incorporated. There are certain criteria that need to be met in order to be incorporated. Aside from a formal 

request, favor is given to content requests that are national in scope. Additionally, new content items requested 

should assist in making policy decisions. A justification must accompany all requests. 
3 

In order to move forward with the design and testing of a new questionnaire, there comes a time to exercise a 

closeout for submitting requests for new content. Recognizing the need to be responsive to the agricultural 

community, USDA-NASS developed a protocol for handling late requests. The protocol consists of a series of 

internal review panels aimed to validate the content request, assess impact of the question, and formulate a 

publication strategy. If accepted, a request is usually limited to a yes/no question within the Practices Section. 

Following is a timeline for content development for the 2012 Census of Agriculture. 

Key dates in the 2012 Census of Agriculture Content Development 

February 4, 2009 - Census of Agriculture Release 

May 28, 2009 - Federal Register Announcement 

August 29, 2009 - Federal Register Closed 

August 31, 2010 - Closeout for Content Test 

April 30, 2011 - Closeout for Census of Agriculture 

July 31, 2011 - Census of Agriculture form finalized for printing 

Responses to the census of agriculture are predominately self-reported, which introduces the risk of misreporting. 

This may be due to misunderstanding or misinterpreting the practice being reported, or its applicability to the 

respondent. To combat this, USDA-NASS limits data published from the Practices Sections to expanded operation 

counts. The screening question and subsequent count of farms provide a snapshot of the scale of an emerging issue 

but more importantly provide a vehicle to identify a population for collecting more detailed data. By establishing the 

population, USDA-NASS is positioned to conduct a sampled survey or complete census of those operations with a 

positive response to the screening question. A census follow-on survey is designed for the specified population and 

provides more reliable data that may inform policy decisions. 

USDA-NASS has staff with expertise in survey and questionnaire design for insight into the question wording. 

Figure 1 describes how the agency works with the data user to develop question wording. This staff will consult 

with the group requesting the content to capture the data needs and use appropriate wording to elicit the desired 

publishable data. If time allows, USDA-NASS will perform testing or cognitive interviews with the new content. 

This testing is critical for understanding respondent’s interpretation of the question and for making appropriate 

modifications to the wording. 

Figure 1. Process of Developing Wording for Screening Questions 

As an example, in the 2007 Census of Agriculture, respondents were asked.....At anytime during 2007, did this 

operation – Generate energy or electricity on the farm using wind or solar technology, methane digester, etc.?

When additional funding was available, USDA-NASS conducted a follow-on special study aimed to collect and 

publish more detailed data about specific renewable energy systems. Any operation that responded affirmatively to 

the yes/no screening question was included in the 2009 On-farm Renewable Energy Production Survey. NASS 

worked with subject matter experts within the industry to formulate relevant questions for the follow-on study. 
4 

Data-Information-Policy Feedback 

The driving force behind informed policy decisions is data. Figure 2 demonstrates the relationship between data, 

information, and policy decisions. It also reflects the interactions between public and private sectors, the vehicles for 

gathering data, and significant factors contributing to gaining valuable information. 

Data

Information Policy 

Decisions

Private

Public

Analysis

Research

Anecdotal Survey Census

4

Experience

Figure 2. Feedback Relationships in Survey Design 

Data 

Data can come from several different sources. Figure 2 identifies three specific sources of data. In general terms, 

anecdotal data (“stories” or cases) are not statistically defensible as they come from a limited number of samples or 

are obtained without a known population. However, anecdotal data often lead to conducting a statistically fortified 

survey that properly represents the entire population. The ability to measure a known population is the key to the 

validity and reliability of survey data. Data that come from a properly designed survey represent the entire 

population and are accompanied by quality measures. Quality measures provide the data user with an understanding 

of data reliability. Survey data may indicate statistically significant trends or conditions in a sample, which in turn 

may justify exploring a topic further in a census. Data from a census, for purposes of this paper the Census of 

Agriculture, provide the most comprehensive data picture because the entire population is questioned , leading to 

less variance and higher confidence in the published responses. 

Information 

Information is tied very closely to data gathering though it can come from experience or observation as well as from 

data. Typically, data are obtained and then analyzed through research to produce information. In this sense, someone 

other than the data provider interprets the data to turn them into usable information. Experience in a disciplinary 

field or practical background with the practice or industry may moderate the way the analysis is conducted and the 

way analysis is interpreted into research results. Analysts and researchers such as economists, geographers, 

sociologists, policy analysts, and industry statisticians play an instrumental role because without them the data are 
5 

effectively just numbers. The data analyst (data user) should be independent from the statistical agency (data 

collector) to insure greater objectivity in analytical results and interpretations. 

 

 

Information may also become available over time and through experience. In many cases however, information 

gained from experience will lead back to data collection for validating new research hypotheses or updating 

previous results. Information from experience alone is rarely as influential in guiding policy decisions as when 

accompanied with sound data analysis. 

The relationship between data and information forms a loop. After data are collected, analyzed, and transformed into 

information there are often gaps which lead to modification of data collection procedures to narrow or close these

gaps. This can occur repeatedly and is enhanced through interactions with data users and data collectors. These 

interactions lead to improved data and information quality and over time provide a series of data that reflects trends. 

Analysis of trends may help describe the longitudinal effects of a policy choice, and lead to policy improvements. 

Policy Decisions 

Policy decisions are often based on information provided but frequently a question about the appropriate policy 

choice drives the need for data. Public and private policy makers may not have sufficient information, which in turn 

may be due to lack of data necessary for analysis. Unsatisfied demand for information by a policy maker 

(information user) can result in support for new data collection efforts that will ultimately provide information to 

satisfy this demand. From a purely scientific standpoint, policy decisions are ideally made after data is collected and 

turned into usable information that answers the policy questions raised. However, this may not occur. Sometimes, 

data is gathered ex post as a way of assessing the impact of the policy. Sometimes the public and private decision 

makers will request data in order to amend an existing policy decision. Sometimes the private and public sector 

interact to identify needed information and support additional data collection. 

An Organic Example 

Organic data collected though the Census of Agriculture provides a good example of how the Data Cycle works. 

Table 1 shows some of the significant events that led to organic content additions to the census of agriculture. 

Table 1. Industry-Policy Timeline of Events Leading to Organic Data Collection in the Census of Agriculture 

1980’s 1990’s 2000’s

*1980 – USDA Study Team on 

Organic Farming issues report and 

recommendations 

*1984 – Organic Food Production 

Association of America founded. 

Later becomes Organic Trade 

Association (OTA) 

*1985 – Food Security Act 

*1988 – Low-input Sustainable 

Agriculture (LISA) programs 

implemented to promote sustainable 

farming practices 

*1990 – Organic Foods Protection 

Act requires USDA to develop and 

maintain National standards 

*1990 – Organic Farming Research 

Foundation (OFRF) founded. 

Conducts national farm surveys in

1993, 1995, 1997,2002 

*1997 – USDA releases proposed 

organic rules for public comment 

*1999 – American Organic 

Standards ratified by OTA 

*1997-1999- Acceleration in 

acquisitions of organic food 

processors by nonorganic firms 

*2000 – USDA releases National 

Organic Program Standards 

*2001 – Accreditation begins 

*2002 – Census of Agriculture 

begins collecting organics data 

*2007 – Census of Agriculture 

increases amount of data collected 

on organic production 

*2009 NASS Conducts Special 

Study on Organic Production Survey 
6 

Beginning with the formation of a USDA Team to study and make recommendation on organic agriculture in 1980, 

policy decisions started to take shape. Early policy decisions set the foundation for advancement in research and 

food safety. By the 1990’s, the public sector was directed to establish guidelines for producers to be certified as 

organic. Meanwhile, the private sector recognized a need to have data for making policy decisions. By the end of the 

decade, the private sector was pushing the public sector for organic qualifications because of consumer driven sales 

which were now crossing state boundaries. 

The organic sector formed its own data gathering organization and conducted national surveys of farmers to fill the 

void of missing public data on organic production and marketing. Using certifier registries and appealing to farmers 

to respond in their own self-interest, the collected data were not representative of the entire U.S. organic farming 

sector until reliability and validation standards were applied in the 1997 and 2002 surveys. At the same time, the 

USDA-ERS was creating estimates of amount of organic acreage by commodity using certifier records. It wasn’t 

until the 2000’s that accurate data could be collected based on the public sector’s insistence on accreditation for all 

organic farmers to a single standard, which defined the population. 

The decision to establish a known population led to data being collected in the 2002 Census of Agriculture in the 

form of a single question relating to organic commodity sales value. The initial data did not provide enough usable 

information for detailed analyses. However, the results helped drive interest in expanding content in the 2007 

Census of Agriculture. Figure 3 shows the content evolution between the 2002 and 2007 Censuses of Agriculture. 

2007

2002

Figure 3. Evolution of Organic Content in Census of Agriculture from 2002 to 2007 

With a now well established population, USDA-NASS was able to capture more detailed data with a Census

Follow-on in 2009. USDA-NASS worked with groups representing both the public and the private sectors to design 

the follow-on study. In 2012 the organic content in the Census of Agriculture references the National Organic 

Program as a result of comments by data users that the self-reporting aspect of the census allowed noncertified 

organic practitioners to be counted without the distinction being made. Policy makers could not accurately 

distinguish if programs were having the desired effects and data users could not produce statistics by the various 

categories of producer certification. Figure 4 shows how the organic content in the latest Census of Agriculture 

addresses these concerns. 
7 

2012

Figure 4. Organic Content in 2012 Census of Agriculture 

In 2008, with a well-established population from which to sample, USDA-NASS was able to capture more detailed 

data with an organic follow-on survey. The results of the 2008 Organic Production Survey were well received and 

led to additional collaboration with other public sectors to guide policy decisions. Subsequently, new policy needs 

arose as the USDA Risk Management Agency (USDA-RMA) moved toward developing organic crop insurance 

premiums. A collaboration between USDA-NASS and USDA-RMA resulted in the 2011 Certified Organic 

Production Survey, another inter-census survey designed to collect information for a government program that 

benefits organic producers. Figure 5 compares portions of the two follow-on surveys related to organic farming that 

drew from the 2007 Census of Agriculture sampling frame. 

2008 2011 

Figure 5. 2008 and 2011 Inter-Census Organic Production Surveys 
8 

Growing Interest in Locally Sourced Food 

Interest is growing in supporting local agricultural economies through purchase of foods from sources that are 

geographically close to the consuming areas, via channels at are direct from farm to consumer or at most one step 

removed. A significant policy push for local food systems occurred with the institution of the USDA Know Your 

Farmer, Know Your Food Initiative (KYF2) in September 2009. The KYF2 Initiative was in turn a response to the 

perceived consumer and producer interest. The initiative was designed to eliminate organizational barriers to 

improve coordination and availability of resources for the promotion of local food systems. Johnson, Cowan, and 

Assenberg (2012) noted that many community and farm advocacy groups are pushing for changes in the next major 

agricultural program legislation (the Farm Bill) that will directly target local foods producers, consumers, and 

markets. Despite the emphasis of policy makers, there are few national surveys of local production and sales that 

provide the necessary sampling frame for conducting a representative and reliable national estimate of production 

and consumption.

Low and Vogel (2011) reported that while mainstream markets account for $297 billion in sales, direct producer-toconsumer markets are worth $1.2 billion, while intermediated (producer-to-intermediary-to consumer) markets are 

estimated to generate $2.7 billion in sales. The USDA-ERS ARMS used to produce these estimates had no 

benchmark data except for questions on the Census of Agriculture that asked respondents to indicate whether the 

operation had sales of products direct to consumers for human consumption and the value of those sales. In this 

case, the policy decisions were driving the data collection, which was at the level of a survey. The data were used to 

produce analysis and research results that in turn were supporting policy decisions. Yet, this work did not have a 

supporting framework of a defined national population from which to draw the survey sample. 

Public sector and private sector policy decisions in support of local food systems are driving an intensifying need for 

data. For example, the Tester Amendment to the 2011 Food Safety and Modernization Act of 2011 exempts local 

sales and direct marketing from certain provisions in an effort to reduce regulatory barriers to transactions. An 

exemption from federal government inspection of meat sold across adjacent state boundaries if inspected to federal 

standards by state agencies gives greater control to state governments to promote regional meat sales. Market 

promotions of locally sourced foods by private sector sellers from farmers market managers to Walmart is 

increasing the need for information about where that food will come from. 

As a direct response to this policy decision push, the USDA-NASS included a content question in the 2012 Census 

of Agriculture to capture data needed to benchmark the size of the intermediated local foods market. Figure 6 shows 

the relationship between the direct to consumer sales and the intermediated market sales questions. 

Figure 6. Positioning of Direct Sales and Intermediated Sales Questions in the 2012 Census of Agriculture 
9 

The intermediated sales question is in the Practices Section, where it can be used in conjunction with the direct sales 

questions to develop a population sampling frame for a follow-on survey of producers who sell through these 

marketing channels. Lessons learned from the Organic Production Survey and input from data users and policy 

makers have helped to streamline the development of the question and its placement in the 2012 Census of

Agriculture. 

Conclusions 

Demands for data to analysis special populations of agricultural producers are growing. The most visible national 

interest currently is to obtain data that will support development and assessments of local food systems programs 

and policies. The dual push of public and private decision makers for new content in the Census of Agriculture has 

been observed previously in response to dramatic growth in organic production and consumption. As a result of selfanalysis and experience with the organic data demands, USDA-NASS has refined its procedures for introducing new 

content, developing population frames, and getting subpopulation reports out to the public and private sector users 

who need the information for decision making. Responsiveness to the needs of policy makers while maintaining the 

integrity of the process, the reliability of the data, and the usefulness of the delivered products remain the goals of 

USDA-NASS in constructing and implementing the Census of Agriculture. 

References

Johnson, R., C. Tadlock, and R.A. Aussenberg. 2012. The Role of Local Food Systems in U.S. Farm Policy. CRS 

Report for Congress R42155, Congressional Research Service, Washington, DC. 

Low, S., and S. Vogel. 2011. Direct and Intermediated Marketing of Local Foods in the United States. ERR-128. 

USDA Economic Research Service, Washington, DC. 

The Council on Food, Agricultural, and Resource Economics (C-FARE). 2007. Improving Information About 

America’s Farms and Ranches: A Review of the Census of Agriculture. C-FARE, Washington, DC.