United States 

 Department of 

 Agriculture

 

 

 

 National 

 Agricultural 

 Statistics 

 Service

 Research and 

 Development Division 

 Washington DC 20250 

 RDD Research Report 

 Number RDD-xx-xx 

 January 2008 

Nonresponse in Phase 

III of the Agricultural 

Resource Management 

Survey in Louisiana 

Michael W. Gerling 

HoaiNam N. Tran 

Morgan S. Earp 

Sammye Crawford 

This paper was prepared for limited distribution to the research community outside the United States 

Department of Agriculture. The views expressed herein are not necessarily those of the National 

Agricultural Statistics Service or of the United States Department of Agriculture. 

 United States 

 Department of 

 Agriculture

 

 National 

 Agricultural 

 Statistics 

 Service

 Research and 

 Development Division 

 Washington DC 20250 

 RDD Research Report 

 Number RDD-08-07 

 June 2008

i 

EXECUTIVE SUMMARY 

The National Agricultural Statistics Service (NASS) surveys farmers and ranchers in the fifty 

states and Puerto Rico to estimate crops and livestock, assess production practices, and identify 

economic trends. The Agricultural Resource Management Survey (ARMS) collects data 

covering chemical use and production practices with a specific focus on the financial well-being 

of agricultural operations. ARMS is composed of three phases. Phase I is conducted May 

through July and screens for potential inclusion in Phases II and III. Phase II is conducted 

October through December and collects data on cropping practices and chemical usage. Phase III 

occurs February through April of the following year and collects detailed economic information 

about the agricultural operation and the operator’s household. ARMS data are used by farm 

organizations, commodity groups, agribusinesses, Congress, state departments of agriculture, and 

the United States Department of Agriculture (USDA). The USDA uses ARMS data to evaluate 

the financial well-being of farms and ranches and to objectively evaluate critical issues related to 

agriculture and the rural economies. 

ARMS Phase III (ARMS III) is the only phase of the ARMS with response rates generally lower 

than 80 percent. The lower response rate for this phase causes a higher risk of biased results. To 

improve ARMS III response rates, NASS’ Louisiana Field Office and NASS’ Research and 

Development Division, conducted two studies. 

The first study assessed the effect of providing an economic brief “Economic Well-Being of 

Farm Households” to a random sample of operations. The economic brief, which was produced 

by the USDA’s Economic Research Service, summarizes data from past ARMS and evaluates 

the financial performance of U.S. farms and ranches. NASS hoped to demonstrate the 

importance of completing the ARMS survey by providing operations with an example of how the 

data are used and thus increase the operators’ willingness to respond. An experimental design 

was developed to evaluate the effect of the economic brief on response rates. Operators were 

randomly assigned to either the treatment group or the control group. Prior to ARMS III data 

collection, the treatment group received a pre-survey letter announcing the survey and a copy of 

the economic brief. The control group received only the pre-survey letter. The analyses showed 

that response rates were not significantly affected by providing the economic brief prior to data 

collection. 

The second study ran simultaneously, and its purpose was to enable NASS to better understand 

nonresponse in ARMS III. Field enumerators asked sampled operators who had declined to 

cooperate on the survey to explain why they were refusing to complete the ARMS III 

questionnaire. Their reasons for refusing were then classified using an updated listing of refusal 

reasons originally created in a previous NASS study, Identifying and Classifying Reasons for 

Nonresponse on the 1991 Farm Costs and Returns Survey (O’Connor, 1992). Enumerators also 

recorded reasons for identifying sampled operations as inaccessible. The tables located on the 

next page display the primary reasons for nonresponse. 
ii 

The authors recommend that the nonresponse study be expanded to other surveys and states. 

This will provide NASS with documented reasons for survey nonresponse and will aid NASS in 

determining what areas of the data collection process (questionnaire design, field enumerator 

training, promotion and public awareness of NASS surveys) need improvement. 

Top Five Reasons for Nonresponse (Refusal & Inaccessible/Incomplete) 

Reasons for Refusal Reasons for 

Inaccessible/Incomplete 

1. Would not take time / too busy. 1. 

Tried several times; could not reach 

anyone for an appointment. Just an 

extremely busy person. 

2. Information too personal / none of 

your business. 2. 

INCOMPLETE – Respondent 

provided partial information, but 

would not or could not provide 

enough information to make the 

questionnaire complete. 

3. Refused, but no reason given. 3. Inaccessible, but no reason given. 

4. I will have nothing to do with the 

Government. 4. Respondent postponed the interview beyond the end of the survey period. 

5. Will do other surveys, but not 

financial surveys. 5. No respondent, as listed on the label, could be found. 
iii 

RECOMMENDATIONS 

1. Field offices should consider whether or not to distribute Economic Research Service’s 

provided economic brief, “Economic Well-Being of Farm Households”, to sampled 

operators. Although this was provided to all ARMS III participating field offices for 

distribution, the research in Louisiana showed that the incentive had no effect on 

response rates for ARMS III. 

2. Continue researching the reasons for nonresponse in Louisiana to study any trends that 

are occurring. 

3. Gradually expand nonresponse research to all surveys and states. This will improve 

NASS’ understanding of survey specific, state, regional, and national nonresponse 

trends. 

4. Apply the lessons learned from this study to future nonresponse 

 studies. Specifically, implement the following: 

a. An office use box on the front or back page of the questionnaire should be 

designated for nonresponse research. 

b. An edit check should be written that triggers a warning if the questionnaire is 

coded refusal, inaccessible or incomplete and is missing a nonresponse reason 

code. 

5. As NASS’ knowledge of nonresponse grows, improve current training scenarios to 

address the major reasons for refusals and inaccessibles. 
iv 
1 

Nonresponse in Phase III of the 

Agricultural Resource Management Survey 

in Louisiana 

Michael W. Gerling, HoaiNam N. Tran, Morgan S. Earp 1

Sammye Crawford2,3

Abstract 

The National Agricultural Statistics Service (NASS) surveys farmers and ranchers 

in the fifty states and Puerto Rico in order to estimate crops and livestock, assess 

production practices, and identify economic trends. One of NASS’ annual 

surveys is the Agricultural Resource Management Survey (ARMS). ARMS 

collects data on chemical use and production practices with a specific focus on the 

financial well-being of agricultural operations. 

ARMS is composed of three phases with Phase III being the only phase with 

response rates typically lower than 80 percent. This low response rate causes a 

higher risk of biased results. To improve Phase III response rates, NASS’ 

Louisiana Field Office and NASS’ Research and Development Division, 

conducted two studies. The first study assessed the effect of providing USDA’s 

Economic Research Service’s economic brief, “Economic Well-Being of Farm 

Households” to a random sample of operations. The second study assessed 

reasons for questionnaires being coded as refusals or inaccessibles. 

Key Words: Incentives, Nonresponse, Response Rate, Refusals, Inaccessibles

 

1 Michael W. Gerling , HoaiNam N.Tran and Morgan S. Earp are Mathematical Statisticians with 

 the National Agricultural Statistics Service – Research & Development Division, located at 3251 

 Old Lee Highway, Room 305, Fairfax, VA 22030. 

2 Sammye Crawford is the Deputy Director of the NASS Louisiana Field Office, located at 5825 

 Florida Boulevard, Baton Rouge, LA 70806. 

3 Special thanks to NASS’ Louisiana Field Office, Field Enumerators, Colby Tolbert of the 

 Louisiana Field Office, Rich Hopper (Survey Administration Branch), and Terry O’Connor 

 and Christopher Taylor of the Research and Development Division for making this report 

 possible. 
2 

1. INTRODUCTION

The mission of the United States 

Department of Agriculture’s (USDA) 

National Agricultural Statistics Service 

(NASS) is to provide timely, accurate and 

useful statistics on United States and Puerto 

Rico agriculture. NASS conducts hundreds 

of surveys a year to estimate crops and 

livestock, explore production practices, and 

identify economic trends. 

The Agricultural Resource 

Management Survey (ARMS) is a multipurpose national survey conducted annually 

on varying commodities. ARMS is the 

primary vehicle by which NASS obtains 

data on chemical use and production 

practices for target commodities. ARMS 

also collects economic information used for 

assessing the financial well-being of the 

United States’ agricultural sector. 

Over time the response rates for 

ARMS and other NASS surveys have been 

decreasing. To help counteract this 

decrease, NASS is conducting research to 

improve its understanding of the reasons for 

survey nonresponse. This will provide the 

Agency with the necessary tools for 

assessing and improving the data collection 

process. 

 This report focuses on two studies 

conducted in Louisiana. The first study 

examines the effect on response rates of 

providing farm operators with an economic 

brief created from 2005 ARMS data. By 

showing the use and importance of the data 

and the operators’ participation, NASS 

hoped to increase response rates for Phase 

III. The second study focused on examining 

the reasons for nonresponse in Louisiana.

1.1 ARMS

ARMS data are used by farm 

organizations, commodity groups, 

agribusiness, Congress, state departments of 

agriculture, and the USDA. The USDA uses 

ARMS data to evaluate the financial wellbeing of farms and ranches. The USDA also 

uses the data to objectively evaluate 

government policies and other critical issues 

related to agriculture and the rural economy. 

The ARMS is composed of three 

phases. Phase I is conducted May through 

July. This first phase is designed to ensure 

that operations are in business and to 

determine whether they are currently 

producing targeted commodities (i.e. rice, 

soybeans, or broilers). 

Phase II, conducted October through 

December, focuses on chemical usage and 

production practices for target commodities. 

Phase III occurs February through 

April of the following year. ARMS Phase 

III’s (ARMS III) data are used to assess the 

financial condition of the farm sector. This 

phase collects data on income, expenses, 

assets, debt, and operator characteristics. 

1.2 ARMS III

This report focuses on the 2006 

ARMS III in Louisiana, where data 

collection actually occurred in February 

through April of 2007. Operations sampled 

in ARMS III were personally interviewed by 

field enumerators using paper 

questionnaires. Louisiana had four 

questionnaire versions. Three versions were 

commodity specific and one was a general 

version. Responses during the screening 

phase had determined the questionnaire 

version each operation received in ARMS 

III.
3 

1.3 NONRESPONSE 

In general, survey nonresponse 

negatively impacts data and estimates, and it 

increases survey costs. Nonresponse in 

ARMS III affects NASS’ income/expense 

estimates, increases data collection time, and 

significantly complicates the data editing 

process. Nonresponse may also introduce 

bias that can not be easily assessed. 

Therefore, NASS is focusing its efforts on 

understanding and improving the data 

collection process in order to reduce survey 

nonresponse. 

1.4 RESEARCH PROJECT 

ORIGINATION 

In the summer of 2006, NASS’ 

Louisiana Field Office (LA FO) initiated a 

survey nonresponse research project with 

NASS’ Research and Development Division 

(RDD). The original goal was to determine 

whether or not providing an economic brief, 

developed using past ARMS data, would 

increase survey response rates for ARMS 

III. The study was later expanded to 

include the recording/documenting of the 

actual reasons for survey nonresponse. 

NASS expected the following: 

1. Supplying the sampled operators 

with an economic brief based on last 

year’s data would increase survey 

response rates and reduce item 

nonresponse in the Farm Household 

Section of the questionnaire. 

2. Documenting reasons for 

nonresponse would provide a deeper 

understanding of nonresponse as it 

relates to ARMS III, which would, in 

turn, provide valuable insight in 

increasing future response rates. 

2. THE RESEARCH PROJECT 

For the 2006 ARMS III, 617 

operations were sampled in Louisiana, based 

on income, acreage, and known target 

commodities. Louisiana’s target 

commodities were rice, soybeans and 

broilers. The probability of being sampled 

varied based on the operation’s size and its 

commodities. Four versions of the ARMS 

III were used to collect data: a general 

version called the Cost and Returns Report 

(CRR) and three commodity specific 

versions (rice, broilers, and soybeans). 

A pre-survey letter was sent 

informing operators that field 

representatives would be contacting them to 

schedule an interview. Enclosed with the 

pre-survey letter was an economic brief 

produced by the Economic Research Service 

(ERS) titled, “Economic Well-Being of Farm 

Households”. This was provided to all 

NASS ARMS III participating field offices 

for distribution, to show the value of the 

data collected the previous year. [See 

Appendix A for a copy of the economic 

brief.] 

The effect of the economic brief on 

response rates was assessed using an 

experimental design where operators were 

randomly assigned to a treatment or control 

group. Half (308) of the sample received 

the brief. This group was labeled the 

“treatment group”. Those not receiving the 

brief were labeled the “control group”. 

Questionnaires for operations receiving the 

economic brief had a code value of 1 entered 

in the 0093 office use box on the back page. 

This allowed for easy tracking of the group 

to which the questionnaire was assigned. 
4 

After the survey, the response rates of the 

two groups were compared, and differences 

were tested using the Chi-Square Test of 

Independence. 

In the past, NASS has conducted 

various studies on the effect of both 

monetary incentives and nonmonetary ones 

(clocks, hats, etc.) on ARMS III response 

rates (McCarthy, Beckler, & Ott, 2006; 

Beckler, Horvath, & Ott, 2005). However, 

for this study the incentive is an actual 

document that was created from data 

collected in the prior year’s survey. 

There are three types of 

nonresponse: refusals, inaccessibles, and 

incompletes. Refusals are operators who 

were actually contacted for data collection 

but refused to respond. Inaccessibles are 

operators who were not available and thus 

were not contacted for data collection. 

Incompletes occur when, for some reason, 

the field enumerator was not able to 

complete all the questions on the 

questionnaires. In a refusal situation, the 

field enumerator asked the operator why 

he/she chose not to participate in the survey 

and recorded the reason on the 

questionnaire. In the case of an inaccessible 

operator, the field enumerator recorded the 

reason why the operator could not be 

contacted. For incompletes, the field 

enumerator would record why the 

questionnaire could not be fully completed. 

In all cases, the field enumerator 

reviewed a handout containing various 

refusal/inaccessible reasons. Each reason 

had its own corresponding numerical code. 

The field enumerator then recorded the 

appropriate nonresponse code in the 0009 

Office Use Box of the questionnaire. [See 

Appendix B for a copy of the handout.] If a 

reason did not have a corresponding 

response code, the staff in the LA FO would 

allocate a new code number for that reason. 

2.1 ENUMERATOR TRAINING 

Survey workshops are conducted 

annually to train field enumerators on 

ARMS III data collection procedures. 

During the LA FO’s survey workshop, field 

enumerators were informed that only half of 

the ARMS III sample was receiving an 

economic brief. Field enumerators were 

also instructed on the use of the 

supplemental nonresponse hand-out. This 

additional training, in which 30 enumerators 

participated, took approximately 20 minutes. 

2.2 FIELD ENUMERATION 

Of the 617 operations sampled, only 

one questionnaire, where the operator is a 

long time refusal, was held in the office and 

not enumerated. The remaining 616 

operations were utilized to assess the effect 

of the economic brief on response rates and 

to explore the reasons for survey 

nonresponse. 

2.3 PROJECT COSTS 

Additional field enumerator training 

costs were not incurred, since the research 

project-specific training was absorbed into 

the general survey workshop training. No 

additional burden fell on the field 

enumerators since recording the reasons for 

nonresponse is customary for all surveys. 

The economic briefs used for the treatment 

group were provided free of charge by ERS. 

Finally, no additional mailing costs were 

incurred by including the brief with the presurvey letter. 
5 

3. RESULTS 

 The first study examined the effect 

of ERS’ economic brief on response rates. 

The second study explored the most frequent 

reasons for nonresponse. 

3.1 STUDY 1: RESPONSE RATES

Table 1 shows Louisiana’s response rates for 

ARMS III. The terms used throughout the 

report are defined below: 

Usable: Reports with complete data. 

Out of Operation was not operating

Business: during the survey reference 

period. 

Refusal: Chose not to participate in 

the survey. 

Inaccessible: Field enumerator unable to 

 make contact. 

Incomplete: The questionnaire could not 

be fully completed. 

(Generally grouped with 

Inaccessibles.) 

Office Hold: Questionnaire was held in the 

 office and not enumerated. 

Nonfarm: Operation failed to meet 

 NASS’ definition of a farm. 

 Thirteen percent of respondents 

refused to complete the questionnaire and 8 

percent were deemed inaccessible or 

incomplete by field enumerators. 

Table 1: Overall Response Rates

617 in Sample Usable Out of 

Business Refusal Inaccessible or Incomplete 

Office 

Hold 

NonFarm 

Frequency 462 22 82 49 1 1 

Percent1 74.3 3.5 13.2 7.9 <0.2 <0.2 

1 Totals may be over/under 100% due to rounding. 

Table 2 displays the number of 

questionnaires deemed usable, out of 

business, refusal, inaccessible/incomplete, 

office hold, and nonfarm by questionnaire 

version. 

 Table 3 displays the response rates 

by questionnaire version. A Chi-Square 

Test of Independence was conducted to 

compare response rates of the commodity 

specific questionnaires to the Cost and 

Returns Report (CRR) questionnaire. The 

number of refusals significantly varied 

across the treatment and control groups (X2

= 39.21, df = 15, p = 0.0006). Thus, 

response rates varied significantly by 

questionnaire version, due largely to the 

substantially lower usable rate for the CRR. 
6 

Table 2: Response Counts by Questionnaire Version 

Questionnaire 

Version Usable Out of 

Business Refusal Inaccessible or Incomplete 

Office 

Hold 

NonFarm Total 

CRR1 187 18 53 26 0 1 285 

Rice 130 1 10 14 0 0 155 

Soybeans 84 1 12 5 1 0 103 

Broilers 61 2 7 4 0 0 74 

Total 462 22 82 49 1 1 617 

1 Cost and Returns Report 

Table 3: Response Rates by Questionnaire Version (Percent) 1

Questionnaire 

Version Usable Out of 

Business Refusal Inaccessible or 

Incomplete 

Office 

Hold 

NonFarm 

CRR 66 6 19 9 0 <1 

Rice 84 1 6 9 0 0 

Soybeans 82 1 12 5 1 0 

Broilers 82 3 9 5 0 0 

Statistical Test Results: X2 =39.21, df = 15, p=0.0006

1 Totals may be over/under 100% due to rounding. 

This pattern of differences in 

response rates between the CRR version and 

the crop specific versions occurs every year 

across all participating states. This occurs 

for several reasons, some of which reflect 

public relations efforts that are done for the 

commodity specific versions. First, NASS 

met with the national rice and soybean 

organizations to obtain their support. 

Second, NASS also conducted presentations 

to producer groups and organizations on 

how the cost per acre to produce rice and 

soybeans is derived and how this 

information benefits the farmer. Third, for 

broilers, there were bio-security concerns, 

requiring additional communication with the 

national and local broiler organizations and 

contractors to obtain their support and/or 

permission to approach the contractees. 

These contacts generated “buy-in” for the 
7 

commodity specific surveys. However, 

generating “buy-in” for the CRR version is 

difficult since its audience is broadly based. 

Thus CRR promotions had to be geared 

towards all farmers. 

 Finally, in 2006, Louisiana had a 

total of 22,735 farms in their survey 

sampling population for ARMS III, (2,085 

soybean producers, 1,287 rice producers and 

444 broiler growers). The LA FO staff 

theorized that these small numbers of rice 

and soybean producers are surveyed more 

often than the average Louisiana farmer. 

This allowed field enumerators to develop a 

closer working relationship with these 

particular farmers and thus may explain the 

lower refusal rate. This close working 

relationship also allowed operations’ 

business statuses (in/out of business) to be 

determined more quickly, and those 

operations not qualifying for the survey to 

be screened out before sampling occurred.

3.2 ECONOMIC BRIEF ANALYSES 

In an effort to improve response 

rates, the Louisiana Field Office mailed an 

economic brief, “Economic Well-Being of 

Farm Households”, along with a pre-survey 

letter to half of the sample. The economic 

brief, which was developed by ERS, 

summarized data collected from past 

Agricultural Resource Management 

Surveys. The other half of the sample 

served as a control group and received only 

the pre-survey letter. 

 Table 4 shows the response rates 

for each group. Twenty-four operations were 

excluded since they were deemed unusable 

(out of business, office hold, and nonfarm 

operations). 

 Response rates did not significantly 

differ between those who received the 

economic brief and those who did not (X2

 = 

0.38, df = 2, p = 0.82). Therefore, one can 

conclude that the economic brief did not 

have an impact on the overall response rates 

for ARMS III in Louisiana. 

 

Table 4: Economic Brief’s Impact on Response Rates 

Questionnaire 

Completion Type 

Received Economic Brief Did Not Receive the 

Economic Brief 

Number Percent Number Percent 

Usable 232 77 230 78 

Refusal 44 15 38 13 

Inaccessible 24 8 25 9 

Statistical Test Results: X2 =0.38, df = 2, p=0.82 
8 

3.3 ECONOMIC BRIEF’S IMPACT 

 ON SECTION J (FARM 

 HOUSEHOLD) 

 The LA FO also wanted to examine 

whether completeness of Section J (Farm 

Household) of the questionnaire improved 

for those receiving the brief. A copy of 

Section J is included as Appendix C. 

Section J contains questions pertaining to 

farm household demographics: race, 

education, income, farm assets/debt, and 

farm expenses. Due to the personal nature 

of these items, farmers often find this 

section invasive, and thus refuse to answer 

any or all of the questions. Of all the 

sections, Section J has the lowest item 

response rate. 

 If the respondent is unable or refuses 

to answer this section, the office staff enters 

a value of “1” in Office Use Box 2002. 

Only four questionnaires had this section 

coded a complete refusal. Two were from 

the treatment group and the other two from 

the control group. Although these are 

extremely limited counts, it appears 

operators who received the economic brief 

were just as likely to refuse Section J as 

those who did not. However, additional 

states would need to be studied to truly 

assess the effect of the economic brief on 

Section J. 

If Section J was partially complete, 

office staff coded each question left 

unanswered with a “-1”. The analysis 

revealed that the response rate to questions 

in Section J was slightly higher for operators 

not receiving the brief. 

Table 5: Section J (Farm Household) Results 

Minus One “-1” Counts in Section J Received 

Economic Brief 

Did Not Receive 

Economic Brief 

Number of Usable Questionnaires 232 230 

Number of Questionnaires with -1’s in Section J 67 53 

Total Number of -1’s in Section J 416 314 

Average Number of -1’s of those having at least one -1 

in Section J. 6 6 

Median Number of -1’s of those having at least one -1 

in Section J. 1 2 

Percent of Usable Questionnaires having -1 in Sect. J 29% 23% 

4. STUDY 2: REFUSALS AND 

INACCESSIBLES 

 The second study examines the 

reasons behind nonresponse. 

4.1 REASONS FOR REFUSALS 

 When an operator refused to 

participate in the ARMS III, Louisiana’s 

field enumerators were instructed to write 

the reason for the refusal on the 

questionnaire. Next, using the nonresponse 
9 

coding sheet provided, enumerators 

recorded the code that most closely matched 

the reason expressed. 

 The results are displayed in Table 6. 

The primary reason for refusing was “Would 

not take time / too busy.” This was followed 

by “Information too personal / none of your 

business” and “Refused but no reason 

given.” 

There were six questionnaires for 

which the reason expressed for the refusal 

was “I will have nothing to do with the 

Government”. This is unusual since a 

completed Phase I questionnaire is required 

to be eligible for Phase III. However, closer 

examination revealed that the spouse, or 

someone other than the operator, completed 

the screener questionnaire in five of these 

instances. The sixth instance occurred when 

the operator refused to participate but the 

field enumerator filled in Phase I’s 

questionnaire using data from the local Farm 

Service Agency office. 

Table 6: Reasons for Refusing to Participate in ARMS III (by descending frequency order) 

Frequency Percent1/ Reason for Refusal 

23 28 Would not take the time / too busy. 

10 12 Information too personal / none of your business. 

10 12 Refused, but no reason given. 

6 7 “I will have nothing to do with the Government.” 

6 7 Will do other surveys, but not financial surveys. 

3 4 The respondent feels the operation’s records are inadequate to 

complete the interview. 

3 4 Respondent only does compulsory surveys. 

3 4 Contact attempted, but respondent refuses on all surveys, and 

refused on this one. 

2 4 Family illness / death. 

2 2 The respondent feels that surveys and reports hurt the farmer more 

than help. 

2 2 Would not keep appointments. 

2 2 “My farm is too small to count / too small to be representative. 

1 1 “I just did a different survey for your office.” 

1 1 “I do not like surveys / I do not do surveys.” 

1 1 Does not think the information is kept confidential. 

1 1 Mentions a specific grievance with the state cooperator. 

1 1 “You contact me too often.” 

1 1 Farm records are at the tax advisors / lawyers. 

1 1 Spouse / secretary / etc. will not let the enumerator see the operator. 

1 1 Does not want to report due to legal / financial problems. 

1 1 Quitting farming. 

1 1 “This is not a farm.” 

82 98 Total 

1 Percent total is under 100% due to rounding. 
10 

 In 1990 and 1991, reasons for 

refusals and inaccessibles were studied 

across multiple states in ARMS III, 

(O’Connor 1991 and 1992). In those years, 

ARMS was called the Farm Costs and 

Returns Survey. Table 7 displays 

Louisiana’s top five refusal reasons and 

where these ranked in past studies. The top 

three reasons are the same in this Louisiana 

study as in the 1990 and 1991 studies. 

Table 7: Comparison Ranking of the Reasons for Refusal in Louisiana’s 2006 ARMS 

 III Versus Nonresponse Studies from 1991 and 1992 

2006 

Louisiana 

Ranking 

1991 Study 

Ranking 

1990 Study 

Ranking Reason for Refusal 

1 1 1 Would not take the time / too busy. 

2* 3 3 Information too personal / none of your business. 

2* 2 2 Refused, but no reason given. 

4* 6* 9 “I will have nothing to do with the Government.” 

4* 6* 27 Will do other surveys, but not financial surveys. 

 * Represents a tie. 

4.2 REASONS FOR 

INACCESSIBLE/INCOMPLETE 

 As with refusals, field enumerators 

were instructed to record the reasons for the 

questionnaires being coded inaccessible or 

incomplete. 

 Table 8 displays the results. The 

three main reasons for questionnaires being 

recorded as inaccessible were 1.) “Tried 

several times; could not reach anyone for an 

appointment. Just an extremely busy 

person”, 2.) “INCOMPLETE – Respondent 

provided partial information, but would not 

or could not provide enough information to 

make the questionnaire complete”, and 3.) 

“Inaccessible, but no reason given.” 

There were seven instances of “No 

respondent listed on the label could be 

found”. This is unusual since the 

questionnaire was coded complete in Phase 

I. A closer examination of the data revealed 

that one such instance involved the operator 

relocating the entire operation to another 

state. The other six occurrences involved a 

change in the point of contact (partner 

spouse or someone other than the operator). 

Perhaps field enumerators should be 

instructed to write additional notes to 

explain why the operator on the label could 

not be found. 
11 

Table 8: Reasons for Inaccessible or Incomplete 

Frequency Percent1/ Reasons for Inaccessible or Incomplete 

10 20 Tried several times; could not reach anyone for an appointment. Just 

an extremely busy person. 

9 18 

INCOMPLETE – Respondent provided partial information, but would 

not or could not provide enough information to make the questionnaire 

complete. 

8 16 Inaccessible, but no reason given. 

7 14 Respondent postponed the interview beyond the end of the survey 

period. 

5 10 No respondent, as listed on the label, could be found. 

4 8 Farm records are not available until after the survey period closes. 

3 6 The operator is away on business. 

3 6 Illness / death in the family prevents the operator from responding. 

49 98 Total 

1 Percent total is under 100% due to rounding. 

As stated earlier, nonresponse 

studies for ARMS III have been conducted 

in the past. Table 9, located on the 

following page, shows Louisiana’s top three 

inaccessible/incomplete reasons and their 

ranking in past studies. Louisiana’s 2006 

results are similar to the 1991 study but are 

different from the 1990 study. In 1990, the 

top two reasons nationwide were “The 

operator is away on extended vacation” and 

“Illness/death prevents the operator from 

responding.” However, these differences 

may be misleading since the earlier studies 

covered more states. Therefore, broader 

based nonresponse studies should be 

conducted to see whether the reasons for 

inaccessibles are the same across states. 
12 

Table 9: Comparison Ranking of the Reasons for Inaccessibles in Louisiana’s 2006 

ARMS Phase III Versus Past Studies 

2006 

Louisiana 

Study 

Ranking 

1991 Study 

Ranking 

1990 Study 

Ranking Reasons for being Inaccessible or Incomplete 

1 1 3* Tried several times; could not reach anyone for 

an appointment. Just an extremely busy person.

2 2 10 

INCOMPLETE – Respondent provided partial 

information, but would not or could not provide 

enough information to make the questionnaire 

complete. 

3 10* 8 Inaccessible, but no reason given. 

* Represents a tie.

5. ECONOMIC BRIEF’S IMPACT 

 ON REASONS FOR REFUSING 

 

Results of these two studies were 

combined to determine whether the 

economic brief had an impact on the reasons 

for refusals. Table 10 compares refusal 

percentages by treatment group. A ChiSquare Test of Independence was conducted 

and found that the economic brief had no 

significant effect on the rates of various 

refusal reasons, (X2 = 21.76, df = 21, p > 

.05). 
13 

Table 10: Impact of Economic Brief on the Reasons for Refusal 

Received Economic 

Brief 

Did Not Receive 

Economic Brief 

Reason for Refusal 

Count Percent Count Percent 

14 32 9 24 Would not take the time / too busy. 

5 11 5 13 Information too personal / none of your business. 

5 11 5 13 Refused, but no reason given. 

3 7 0 0 Contact attempted, but respondent refuses on all 

surveys, and refused on this one. 

3 7 3 5 Will do other surveys, but not financial surveys. 

2 5 4 11 “I will have nothing to do with the Government.” 

2 5 0 0 “My farm is too small to count / too small to be 

representative.” 

1 2 2 5 The respondent feels the operation’s records are 

inadequate to complete the interview. 

1 2 0 0 Mentions a specific grievance with the state cooperator. 

1 2 0 0 “I just did a different survey for your office.” 

1 2 0 0 Quitting farming. 

1 2 1 3 The respondent feels that surveys and reports hurt the 

farmer more than help. 

1 2 1 3 Would not keep appointments. 

1 2 0 0 “I do not like surveys / I do not do surveys.” 

1 2 0 0 Farm records are at the tax advisors / lawyers. 

1 2 0 0 Spouse / secretary / etc. will not let the enumerator see 

the operator. 

1 2 0 0 “This is not a farm.” 

0 0 3 8 Respondent only does compulsory surveys. 

0 0 1 3 Does not think the information is kept confidential. 

0 0 1 3 Does not want to report due to legal / financial 

problems. 

0 0 1 3 “You contact me too often.” 

0 0 2 5 Family illness / death. 

44 98 38 99 Total 

Statistical Test Results: X2 =21.76, df = 21, p=0.41 

1 Percent total is under 100% due to rounding. 
14 

6. ECONOMIC BRIEF’S IMPACT

ON REASONS FOR 

INACCESSIBLE/INCOMPLETE 

The effect of the economic brief on 

the reason for being recorded as 

inaccessible/incomplete was examined. The 

total number of inaccessibles/incompletes 

was 49. Table 11 displays the breakdown of 

those inaccessibles based on treatment 

group. Although the top three reasons (see 

Table 8) for inaccessible or incomplete 

questionnaires were used more frequently by 

those who received the economic brief than 

by those who did not, these differences were 

not statistically significant, as shown in the 

Chi-Square Test of Independence results.

Table 11: Impact of the Economic Brief on the Reasons for being 

Inaccessible/Incomplete 

Received 

Economic Brief 

Did Not Receive 

Economic Brief 

Reasons for being Inaccessible or Incomplete 

Count Percent Count Percent 

6 25 4 16 Tried several times; could not reach anyone for an 

appointment. Just an extremely busy person. 

5 21 4 16 

INCOMPLETE – Respondent provided partial 

information, but would not or could not provide enough 

information to make the questionnaire complete. 

5 21 3 12 Inaccessible, but no reason given. 

3 13 4 16 Respondent postponed the interview beyond the end of the 

survey period. 

2 4 4 16 No respondent, as listed on the label, could be found. 

1 8 2 8 Farm records are not available until after the survey period 

closes. 

1 4 2 8 The operator is away on business. 

1 4 2 8 Illness / death in the family prevents the operator from 

responding. 

24 100 25 100 Total 

Statistical Test Results: X2 =2.80, df =7, p=0.90 
15 

7. LESSONS LEARNED 

 For future nonresponse studies to run 

more efficiently and effectively, the authors 

recommend designating a cell on the 

questionnaire in which to capture the reason 

for nonresponse. The authors also 

recommend developing an edit check to 

ensure that a reason code is present for those 

questionnaires coded refusal or inaccessible. 

8. CONCLUSION 

 Mailing the Economic Research 

Service’s economic brief with the presurvey letter had no effect on response rates 

for ARMS III or on the completion rates of 

Section J (Farm Household). 

 Studying the reasons for nonresponse 

has provided the Louisiana Field Office with 

a starting point for addressing future 

nonresponse. As a better understanding of 

nonresponse is obtained, better enumerator 

training scenarios can be developed for 

addressing the major reasons for refusals 

and inaccessibles. 

 Finally, the Research and 

Development Division and the Louisiana 

Field Office will continue working together 

to study nonresponse and how best to 

increase response rates. 

 

9. REFERENCES 

Beckler, D., J. McCarthy, K. Ott. (2006) The 

Effect of Incentives on Response in 2005 

ARMS Phase 3 Interviews, Research and 

Development Division Report RDD-06-06, 

United States Department of Agriculture, 

National Agricultural Statistics Service. 

Beckler, D., P. Horvath, K. Ott. (2005) 

Indirect Monetary Incentives for the 2004 

ARMS Phase 3 Core, Research and 

Development Division Report RDD-05-05, 

United States Department of Agriculture, 

National Agricultural Statistics Service. 

Gerling, M. (2005) Using Handheld Global 

Positioning System Receivers for Phase II of 

the Agricultural Resource Management 

Survey, Research and Development Division 

Report RDD-05-04, United States 

Department of Agriculture, National 

Agricultural Statistics Service. 

O’Connor, T. (1991) Identifying And 

Classifying Reasons For Nonresponse On 

The 1990 Farm Costs And Returns Survey, 

Research and Development Division Report 

SRB-91-11, United States Department of 

Agriculture, National Agricultural Statistics 

Service. 

O’Connor, T. (1992) Identifying And 

Classifying Reasons For Nonresponse On 

The 1991 Farm Costs And Returns Survey, 

Research and Development Division Report 

SRB-92-10, United States Department of 

Agriculture, National Agricultural Statistics 

Service. 

Ott, L. (1988) An Introduction To Statistical 

Methods And Data Analysis Third Edition, 

Boston, Massachusetts: PWS-Kent 

Publishing Company. 
16 

Appendix A 

Economic Research Service’s 

Economic Well-Being of Farm Households 
17 
18 
19 
20 
21 

Appendix B 

Field Enumerator Instructions 

(Includes a Copy of the Listing of Reasons for Nonresponse) 
22 
23 
24 

Appendix C 

Section J (Farm Household) 

of the 

Agricultural Resource Management Survey Phase III Questionnaire
25 
26 
27 
28 
29 
30