United States 

Department of

Agriculture

National

Agricultural

Statistics

Service

Research and

Development Division

Washington DC 20250

RDD Research Report

Number RDD-10-01

May 2010

Does Using a 

Personalized Pre-Survey

Letter Improve the 

Response Rate for the 

June Agricultural 

Survey in Louisiana?

Michael W. Gerling

HoaiNam N. Tran

Sammye Crawford

Darcy Miller

Terry P. O’Connor

This report was prepared for limited distribution to the research community outside the United States 

Department of Agriculture. The views expressed herein are not necessarily those of the National 

Agricultural Statistics Service or of the United States Department of Agriculture. 

i

EXECUTIVE SUMMARY

The United States Department of Agriculture’s (USDA) National Agricultural 

Statistics Service (NASS) surveys the United States’ and Puerto Rico’s

agriculture. The data collected are then compiled to estimate crops and livestock, 

assess production practices, and identify economic trends. The June Agricultural 

Survey (JAS) is conducted by NASS to provide the first clear indication of the 

potential crop production and supply of major commodities for the year. The data 

collected are also used as the basis for several additional follow-on surveys 

throughout the year. 

Louisiana is one of 49 states (Alaska excluded) which conducts the June 

Agricultural Survey. The Louisiana Field Office utilizes personalized pre-survey

letters in an effort to improve response rates by conveying a more personalized

touch to potential respondents. In this context, personalized means that the name

and address appears in the address area of the letter, the name of the operator of 

the agricultural operation appears in the salutation, and a digitized blue ink 

signature of the state director appears in the signature area. The overall 

preparation and mailing processes used are quite labor intensive compared to 

sending out a generic pre-survey letter. The Louisiana Field Office wanted to 

assess whether the additional effort had a return benefit in terms of increased 

response rate versus using a generic pre-survey letter.

To research the efficacy of personalized questionnaires in increasing the response 

rate, NASS’ Research and Development Division worked with the Louisiana 

Field Office on a split sample test on the 2008 June Agricultural Survey.

After analyzing the data, there was no statistical evidence that personalizing the 

pre-survey letters improved the response rate for the June Agricultural Survey.
ii

RECOMMENDATIONS

1. The Louisiana Field Office should discontinue the practice of using 

personalized pre-survey letters if response rate improvements are the 

sole objective, since no positive return was found to offset the 

resources required for personalization of pre-survey letters. 

2. Research should be expanded to additional states to determine if 

using personalized pre-survey letters has any effect on the response 

rate at a national level. 
1

Does Using a Personalized Pre-Survey Letter 

Improve the Response Rate for the June Agricultural Survey in 

Louisiana?

Michael W. Gerling, HoaiNam N. Tran, Sammye Crawford, 

Darcy Miller, Terry P. O’Connor1

Abstract

The United States Department of Agriculture’s (USDA) National Agricultural 

Statistics Service (NASS) surveys farmers and ranchers across the United States

and Puerto Rico in order to estimate crops and livestock, assess production 

practices, and identify economic trends. One of the surveys NASS conducts is the

Agricultural Survey, conducted four times a year, (March, June, September and 

December). June is the base quarter of the survey, and it is the focus of this study. 

In recent years, NASS’ Louisiana Field Office has used personalized pre-survey

letters in an effort to increase the response rate. However, this process is very 

labor intensive compared to mailing a generic pre-survey letter. Given increasing 

workloads, the Louisiana Field Office sought to determine whether the practice 

provided positive return for the time expenditure.

This study examines whether personalized pre-survey letters result in a higher 

survey response rate compared to using generic pre-survey letters.

Key Words: Agriculture, (Personalized / Generic) Pre-survey Letter, Data 

Collection, Response Rate

___________________ 

1/ Michael W. Gerling, HoaiNam N. Tran, and Darcy Miller are mathematical statisticians and

Terry P. O’Connor is Head of the Collection and Dissemination Research Section for the

National Agricultural Statistics Service - Research & Development Division, located at 

3251 Old Lee Highway, Fairfax, VA 22030. Sammye Crawford is the Deputy Director of the 

Louisiana Field Office, 5825 Florida Boulevard, Baton Rouge, LA 70806. The authors would 

like to thank the Louisiana Field Office staff for making this research possible.
2

1. INTRODUCTION

The National Agricultural Statistics 

Service’s (NASS) primary purpose is to 

provide timely, accurate and useful statistics 

on the United States’ and Puerto Rico’s

agriculture. NASS conducts hundreds of 

surveys annually for the purpose of making 

estimates on crops and livestock, exploring 

production practices, and identifying 

economic trends.

The Agricultural Survey occurs four times a 

year, in March, June, September and 

December. June is the base quarter and 

collects information on U.S. crops, 

livestock, grain storage capacity, and type 

and size of farms. June is also the focus of 

this study. 

The 2008 June Agricultural Survey (JAS) 

sample was comprised of 87,151 agricultural 

operations across the United States. 

Louisiana had 1,290 agricultural operations 

sampled.

1.1 Problem: Creation and Mailing of 

Personalized Pre-Survey Letters is 

Labor Intensive

The Louisiana Field Office has employed a 

number of different methods for improving 

their survey response rate. Survey 

nonresponse negatively affects data 

estimates, increases survey costs and data 

collection time, and significantly 

complicates the data editing and 

summarization processes. Nonresponse also 

increases the potential for introducing a bias

into the estimates which cannot be easily 

assessed. 

One of the methods used to attempt to 

increase the response rate is personalizing 

each sampled agricultural operation’s presurvey letter. In Louisiana this is done by 

printing 1) the name and address in the 

address area of the letter, 2) the name of the 

operator of the agricultural operation in the 

salutation, and 3) a digitized blue ink 

signature of the state director in the 

signature area. 

Overall, preparing and mailing a 

personalized pre-survey letter is more labor 

intensive than a generic mailing. 

Combining a generic pre-survey letter with a 

copy of the labeled questionnaire can be 

mechanized using mailing machines. In 

comparison, each personalized pre-survey 

letter has to be manually combined with the 

appropriate, labeled questionnaire to ensure 

that they are mailed together in the correct 

envelope. This process requires an 

additional 12 hours of manual intervention,

which takes time away from employees’

other work. 

1.2 Purpose of the Research

The goal of the pilot study, which focused 

on Louisiana’s 2008 June Agricultural 

Survey, was to determine whether using a 

personalized pre-survey letter would result 

in a better response rate than using a generic 

pre-survey letter.

1.3 Definitions

There are three types of survey nonresponse: 

1) refusals, 2) inaccessibles, and 3) 

incompletes. 

1.) Refusals are operators who were 

not willing to respond or 

participate in the survey.
2

2.) Inaccessibles occur when field 

enumerators are unable to contact 

or reach the agricultural 

operators for data collection. 

 

3.) Incompletes are questionnaires 

for which at least one of the 

questions is not answered.

2. METHOD

The 2008 JAS sample for Louisiana was 

comprised of 1,290 agricultural operations, 

which were randomly divided into four

treatment groups. Treatment groups A and 

B received the personalized pre-survey letter 

and treatment groups C and D received the 

generic letter.

The treatment groups used are defined as 

follows:

Treatment Group A: Operations received a 

personalized pre-survey letter and were 

visited by a field enumerator to complete the 

interview. See Appendix A.

Treatment Group B: Operations received a 

personalized pre-survey letter and were

asked to complete either the enclosed paper 

questionnaire by mail or the survey 

electronically via the Internet. If this initial 

contact did not result in a completed

questionnaire, an office enumerator would

call the operation to obtain the information. 

If a response could still not be obtained in 

this way, a field enumerator would visit the 

agricultural operation to complete the 

questionnaire. See Appendix B.

Treatment Group C: Operations received a 

generic pre-survey letter and were field 

enumerated only. See Appendix C.

Treatment Group D: Operations received a 

generic pre-survey letter and were asked to 

complete either the enclosed paper 

questionnaire by mail or the survey 

electronically via the Internet. If this initial 

contact did not result in a completed 

questionnaire, an office enumerator would 

call the operation to obtain the information. 

If a response could still not be obtained in 

this way, a field enumerator would visit the 

agricultural operation to complete the 

questionnaire. See Appendix D.

There were two constraints applied to the 

randomization process. First, sampled 

agricultural operations tied to multiple 

operations were restricted to treatment 

groups A and D. Second, those agricultural 

operations identified as long term refusals 

were not assigned a treatment group.

Table 1 displays the data collection mode, 

the pre-survey letter version and the 

characteristics and the number of 

agricultural operations in each treatment 

group. Treatment groups A and C had fewer 

operations than treatment groups B and D,

since these operations (in groups A and C) 

were typically ones that requested field 

enumeration in the past or were Louisiana’s 

larger operations for which the field office 

felt that a personal field visit would be best. 
4

Table 1: 2008 June Agricultural Survey in Louisiana: Treatment Groups Defined and 

 Number of Agricultural Operations in Each.

Treatment Group 

Name

Data Collection 

Mode Pre-Survey Letter Type Number of Operations

A Field Personalized 77

B

Mail,

Web,

Telephone,

Field

Personalized 538

C Field Generic 95

D

Mail,

Web,

Telephone

Field

Generic 531

2.1 Analysis Method

The Chi-square test was used to determine if 

there exists a significant difference between 

personalized and generic pre-survey letter 

frequencies. The 95 percent confident level 

was used with an alpha level of 0.05. 

The hypotheses are as follows: 

Null hypothesis: No significant difference 

between treatment groups exists. 

Alternative hypothesis: There is a 

significant difference between the treatment 

groups. 

2.2 Project Costs

Developing the pre-survey letters and 

organizing and recording which sampled 

agricultural operations received a particular 

pre-survey letter consumed a majority of the 

time -- totaling 24 staff hours.

3. FINDINGS

Tables 2 through 4 show the compiled 

results. Table 2 displays the number and 

percentage of questionnaires that were 

recorded as complete, inaccessible or refusal

by treatment group. 
5

Table 2: 2008 June Agricultural Survey in Louisiana: Number of Completes, 

Inaccessibles and Refusals by Treatment Group. 

Treatment 

Name

Data 

Collection 

Mode

Pre-Survey 

Letter Type

Questionnaires

 Complete Inaccessible Refusal Total

 No. % No. % No. % No. %

A

Field

Personalized 63 81.8 7 9.1 7 9.1 77 100.0

C Generic 76 80.0 10 10.5 9 9.5 95 100.0

B Mail,

Web,

Telephone

Field

Personalized 384 71.4 116 21.6 38 7.1 538 100.11/

D Generic 375 70.6 102 19.2 54 10.2 531 100.0

1/ Due to rounding, total does not equal 100 percent.

Table 3 displays the results of the Chisquare analyses conducted in comparing 

treatment groups. Comparing treatment 

groups A and C shows that there is virtually 

no difference in response rate due to the type 

of pre-survey letter used. The Chi-square 

test with 2 degrees of freedom was 

calculated to be 0.96, which is larger than 

the alpha level, 0.05. This means the two 

treatment groups are not statistically 

different in response rate.

Comparing treatment group B with D, 

shows that the type of pre-survey letter used 

had no effect on the response rate. The Chisquare test with 2 degrees of freedom was 

calculated to be 0.15, which is larger than 

the alpha level. This indicates that there is 

no evidence to conclude that these two 

treatment groups are statistically different in 

response rate. 

Table 3: 2008 June Agricultural Survey in Louisiana: Chi-Square Analysis on 

Treatment Groups

Treatment Name Data Collection Mode Pre-Survey Letter 

Type Chi –Square Value1/

A

Field

Personalized 

0.96

C Generic

B Mail,

Web,

Telephone,

Field

Personalized

0.15

D Generic

1/ Two degrees of freedom.
6

Table 4 displays the number and percentage 

of completes, inaccessibles and refusals by 

pre-survey letter type (collapsing across 

treatment group). The Chi-square test with 

two degrees of freedom was conducted on 

pre-survey letter type versus response type. 

The test showed that there was no statistical 

difference in response rate between using a 

personalized pre-survey letter versus a

generic one since the resulted Chi-square 

value of 0.18 is larger than the alpha level.

Table 4: 2008 June Agricultural Survey in Louisiana: Response Type by Pre-Survey 

Letter

Pre-Survey 

Letter Type

Questionnaires Chi –

Square 

Value

 Complete Inaccessible Refusal Total

No. % No. % No. % No. %

Personalized 447 72.8 123 20.0 45 7.3 614 100.1

1/

0.18

Generic 451 72.0 112 17.9 63 10.1 626 100.0

1/ Due to rounding, total does not equal 100 percent.

4. PAST STUDIES

Other research into using a personalized 

letter over a generic pre-survey letter has 

been conducted over the years. Several 

studies occurred in the 1970’s when 

personalization of pre-survey letters was 

fairly new and innovative. However, only a 

few studies have been done in recent years.

In August 2005, “Effect On Survey Response 

Rate Of Hand Written Versus Printed

Signature On A Pre-surveying Letter: 

Randomized Controlled Trial.” was 

published from the University of Oxford, 

United Kingdom. The authors Kristie

McKenzie-McHarg, Lucy Tully, Simon 

Gates, Sarah Ayers and Peter Brocklehurst

tested whether hand signing the pre-survey

letter improved the response rate compared 

to a computer-generated signature. The 

results showed no detectable difference 

between the groups in the time taken to 

respond. 

In the 2005 International Journal of Market 

Research Vol. 47 Issue 4, “The Effect Of 

Pre-surveying Letter Personalization In 

Mail Surveys” by Phillip Gendall of Massey 

University looked at personalization of mail 

surveys to the general public. The study 

found little or no effect of personalization on 

response rates, response speed, item nonresponse, or social desirability bias. Gendall 

suggests that personalization may no longer 

be effective in mail surveys. Nevertheless, 

he stated that as survey-processing 

technology has advanced over the years, that 

it is often more difficult not to personalize 

survey correspondence than to personalize 

it. Gendall went on to say that, unless there 

is a good reason to avoid personalization, 

survey researchers should use it. At worst, it 

will have no effect, but it might have a 

positive effect.
7

In 2007, Don Dillman authored“Mail And 

Internet Surveys: The Tailored Design 

Method. 2nd Edition.” Dillman states that 

“Recent tests of personalized mailings on 

general public samples, each of which used 

four contacts, resulted in response rate 

increases of 5% to 11%.” However, he

went on to say that “The large scale of 

certain government surveys also makes it 

difficult to use certain techniques that are 

acceptable to OMB and that help to assure a 

high response rate. For example, sending 

out tens of thousands of questionnaires 

makes it difficult to employ personalization 

techniques. This difficulty stems less, 

perhaps, from objections to inserting names 

and address into letters, than it does from the 

risks inherent in requiring that two identified 

pieces of mail be matched and inserted into 

the same envelope.” Dillman also mentions

the possibility of personalization having a 

negative effect due to the perceived costs of 

loss of privacy. Overall, he continues to 

support the use of personalization when 

appropriate and possible. He believes that, 

on average, personalization of mail surveys 

has a significant positive effect.

Based on these research studies, there are

limited and conflicting results on whether 

personalizing pre-survey letters improves

the response rate. 

5. CONCLUSION 

Analyzing response data from Louisiana’s 

2008 June Agricultural Survey shows that 

the use of personalized pre-survey letters

compared to generic pre-survey letters made

no statistical difference in the response rate. 

Past studies, conducted outside of NASS, 

both support and counter the use of 

personalized pre-survey letters. 

Overall, the authors recommend that the 

Louisiana Field Office should discontinue 

using personalized pre-survey letters since 

no positive return was found to offset the 

resources required for personalization of 

pre-survey letters. Also, the authors 

recommend research be expanded to 

additional states to determine if Louisiana’s 

results are only isolated to that state. 

Additional research also could include 

examining if particular content and length of 

the pre-survey letter has an effect. 

In the future, Research and Development 

Division will continue to investigate with 

NASS’ Louisiana Field Office and NASS’ 

Survey Administration Branch (responsible 

for managing all of NASS’ surveys) new 

ways to improve response rates and make

current survey processes more efficient.

6. REFERENCES

Dillman, D. (2007) Mail And Internet 

Surveys: The Tailored Design Method. 2nd

Edition, New York, New York, John Wiley 

& Sons Inc.

Gendall, P. (2005) The Effect Of Presurveying Letter Personalization In Mail 

Surveys.” International Journal of Market 

Research Vol. 47 Issue 4, World Advertising 

Research Center.

Gerling, M., H. Tran, M. Earp. (2008)

Nonresponse in Phase III of the Agricultural 

Resource Management Survey in Louisiana, 

Research and Development Division Report

RDD-08-07, United States Department of 

Agriculture, National Agricultural Statistics 

Service.

McKenzie, K., L. Tully, S. Gates, S. Ayers, 

P. Brocklehurst, (2005) Effect On Survey 
8

Response Rate Of Hand Written Versus 

Printed Signature On A Pre-surveying 

Letter: Randomized Controlled Trial, 

United Kingdom, University of Oxford.

Ott, L., (1988) An Introduction To Statistical 

Methods And Data Analysis Third Edition, 

Boston, Massachusetts:PWS-Kent

Publishing Company.
9

Appendix A

Treatment Group A - Pre-Survey Letter
10

Appendix B

Treatment Group B - Pre-Survey Letter (Page 1 of 2 )
11
12

Appendix C

Treatment Group C - Pre-Survey Letter
13

Appendix D

Treatment Group D - Pre-Survey Letter (Page 1 of 2)
14