1 

Testing Contact and Response Strategies to Improve Response in 

the 2012 Economic Census 

Erica Marquette and Michael Kornbau 

U.S. Census Bureau 

4600 Silver Hill Road 

Washington, DC 20233 

Disclaimer: This report is released to inform interested parties of ongoing research and to encourage discussion of

work in progress. The views expressed are those of the authors and not necessarily those of the U.S. Census Bureau. 

Keywords: Contact Strategies, Response Strategies, Economic Census 

Abstract 

While planning the 2012 Economic Census, the Census Bureau tested six various contact and response strategies to 

improve the overall response rate to the Economic Census while attempting to achieve earlier response and 

obtaining more internet responses. The following six tests were conducted during different phases of mailout to 

achieve this goal: 

1. Sending a follow-up mailing containing only a letter to a sample of nonrespondent single-unit companies

who initially received a questionnaire to determine industry classification. 

2. Mailing a questionnaire to small multi-unit companies prior to the Economic Census to obtain better 

contact information. 

3. Mailing an advance letter prior to the initial mailout to single-unit companies in certain low-response 

industries. 

4. Calling nonrespondent single-unit companies with an automated message reminding them to respond to 

their census questionnaire. 

5. Sending only a letter instead of a questionnaire to single-unit companies during the first follow-up mailing. 

6. Sending certified mailings to single-unit companies in certain low-response industries during the second

and third follow-up mailings. 

This paper will explain the objectives used for each test, how the tests were conducted, how the results were 

evaluated, and provide preliminary results. 

I. Introduction 

The Economic Census is conducted every five years (i.e. years ending in 2 and 7) in order to get a measure of 

American business and the economy. Approximately 500 different form types were mailed to nearly 4 million 

businesses in October through December of 2012 in order to collect the information needed to produce relevant 

estimates for the 2012 Economic Census. The type of form and information collected for each business was based 

on their principle business activity as each industry has different information that needs to be collected. 

Respondents were asked to provide a range of operational and performance data for their companies. For more 

information about the Economic Census data see www.bhs.econ.census.gov. 

The focus of the Economic Census includes companies of all sizes in most economic industries with employees. 

How and when a company is contacted varies based on the size of the company. If a company has more than one 

establishment, it is considered a multi-unit company and is automatically included in the initial mailout. If the 

company only has one establishment, it is a single-unit company, and only a sample of the single-unit companies are 

included in the initial malout. Multi-unit and single-unit companies have different characteristics and respond 

differently. For example, multi-unit companies received a form for each establishment. The information collected 

is usually maintained by different departments within a large company and takes more time to complete. In 
2 

comparison, a single-unit company only received the form for the one establishment. Therefore, data collection 

activities were different for the multi-unit and single-unit companies. 

Two requests were sent prior to the initial Economic Census mailout. In November 2011, some single-unit 

companies were sent an industry classification form and in April 2012 multi-unit companies were sent a contact 

exchange form. The initial form mailout was in October 2012 for multi-unit companies and December 2012 for 

single-units. The due date for the Economic Census was February 12, 2013 for all companies. Form follow-ups 

were conducted for single-units in March, April, and May. In comparison, multi-units were followed-up with a 

letter in March, a letter and/or a phone call in April, and a letter or form in May. Additionally, a fourth and final 

letter follow-up was sent in July to both multi-unit and single-unit companies. 

A company could respond to the Economic Census via two response modes: mail or electronic. A mail response 

involved the company filling out the paper form and mailing it back to the Census Bureau. An electronic response 

involved the company filling out and submitting the form via Surveyor for multi-unit companies or via the Internet 

using a Centurion internet application for single-units. Surveyor is software that the company downloads in order to 

report their data electronically. 

While planning for the 2012 Economic Census, it was decided that contact and response strategies should be tested 

to see if response could be improved or maintained while saving costs and receiving more internet responses. 

Research on households has shown that multiple contacts and contacting respondents by a method other than the 

form does not negatively affect response (Millar, O’Neill, and Dillman, 2009). Additionally, the American 

Community Survey (ACS) has tested strategies to help increase response and encourage the internet response 

option. First they found that an additional reminder mailing showed an increase in the overall response (Chestnut, 

2010). Then they tested a push strategy for internet and found that sending out a letter instead of a form increased 

internet response (Tancreto, et al, 2012). The hope was that businesses would follow a similar pattern. 

Although different methods have been used to try to increase response to the Economic Census, no formal testing 

has ever been conducted. Six contact and response strategies were selected based on prior research and previous 

censuses to be tested during the different phases of the 2012 Economic Census mailout period. The following are the 

tests that were conducted: 

1. Industry Classification Refile 

2. Contact Exchange Form 

3. Targeted Advanced Mailing 

4. Reminder Phone Call 

5. Letter-Only First Follow-up

6. Certified Mailing 

The reminder phone call and letter-only first follow-up tests were conducted together using a factorial design. Due 

to a processing issue, an additional test of a letter only follow-up for the third quarter birth single-unit companies 

was added to the list of tests to be evaluated for a total of seven tests. These tests, how they were conducted, and 

preliminary results will be discussed in this paper. 

Each of these tests had its own set of objectives. For many of them, saving costs was an important one due to the 

increasing cost of conducting the Economic Census if the methods stay the same. All Economic Census results are 

as of August 31, 2013 and are not final. The refile results are final as that was sent out the year prior the Economic 

Census. Only some preliminary results will be presented at this time and cost analysis will not be included. 

II. Results Methodology 

When evaluating the results, three different analysis methods were used. The first method was calculating a checkin rate, which represents the percentage of companies in sample who returned their form or submitted electronically. 

The second method was testing the difference in check-in rates across treatments. The last method was evaluating 

the mean response time. Final results for the study will include the unit response rate and a total quantity response 

rate, a measure of the percentage of the estimate derived from reported data. The data for evaluating these response 

rates are not available at this time. Note that all the results given in the paper are unweighted since all the samples 
3 

were selected to represent the universe in question (which changes from test to test), but are not generalizable to the 

population. 

A. Check-in Rates 

The check-in rate for each test was calculated in the following way: 

. 

The check-in rate represents the percentage of sample establishments that returned a form or responded

electronically (with or without data). When calculating the response mode level check-in rate, the denominator for 

all the rates was the same for the given test. The overall rate is approximately the sum of the rates of the different 

modes. These rates may differ because the overall rate includes companies who were considered checked-in, but 

used some other mode. 

B. Testing Check-in Rates 

For all statistical tests performed in this study, a two-sided t-test at an overall α = 0.10 significance level was used to 

determine which comparisons were significantly different. If there was a family of comparisons, the Bonferroni 

method for controlling the family-wise error rate was used. 

C. Mean Response Time 

The mean response time was calculated by taking the average time in days from initial mailout until a response was 

received from a company. The difference was tested by calculating an ANOVA based on the different treatments 

included in the test. 

III. Industry Classification Refile 

The Economic Census Industry Classification Form (also referred to as the refile) was sent to single-unit companies 

a year prior to the mailing of the 2012 Economic Census form in order to help assign the company an industry 

classification code. The goals of this test were to determine if sending only a letter instead of a letter and form in the 

follow-up mailing increased overall and internet response. For the refile, a company could also respond by calling 

the Census Bureau, but they could not respond by phone for the Economic Census. 

A systematic sample based on industry and annual payroll of half of the companies eligible for follow-up received 

the letter-only mailing containing a company’s online login information during the first follow-up verses a letter and 

a form. The number of delinquent companies at the time of follow-up was 29,165. Therefore, 14,583 companies 

received the letter-only and 14,582 companies received the letter and form. 

Table 1 shows the overall and response mode check-in rates for each of the treatments. The results show that the 

overall check-in rate of the letter and form groups were not statistically significantly different. The internet check-in 

rate of the letter group was greater than the form group and the mail check-in rate was greater for those who 

received the form as a follow up. The interesting result was that significantly more companies who received only 

the letter called the help line and responded via that phone call. This shows that the letter-only merely shifted the 

response mode but did not increase overall response, which is a positive thing because sending a letter-only is 

cheaper, as is an internet response. 
4 

Table 1: Industry Classification Refile Check-in Rates 

Treatment 

Response 

Mode 

Letter-only 

(n=14,583) 

Form and 

Letter 

(n=14,582) Difference 

Internet 23.83% 12.48% 11.35%*

Phone 2.99% 1.57% 1.42%* 

Mail 7.36% 19.60% -12.24%* 

Overall 34.19% 33.65% 0.54% 

* represents the difference was significant 

IV. Contact Exchange Form 

A contact exchange form was sent to multi-unit companies about six months prior to the 2012 Economic Census 

mailout. This form asked for the company’s most recent contact information so that it was the most up-to-date for 

the census. The focus of this test was multi-unit companies with less than 1,000 employees. The results of mailing 

contact exchange cards to larger multi-unit companies for the 2007 Economic Census were favorable in obtaining 

improved addresses and in improving response compared to 2002, but no formal testing was done. The test for 2012 

was conducted to see if the small multi-unit companies showed similar improvements in the 2012 Economic Census. 

The goal of this test was to determine if sending the contact exchange card increased overall and internet response to

the 2012 Economic Census. 

The decision to perform this test occurred after the contact exchange forms were printed, so it was decided to 

exclude 2,000 companies from the mailout for testing purposes. A systematic sample of 2,000 based on industry 

and annual payroll of the small multi-unit companies was selected to essentially be the control group that would not 

receive the contact exchange form treatment. 

Table 2 shows the check-in rates for the different treatments. The difference in the overall check-in rates is 

currently not significantly different, as are the differences in the mode level check-in rates. Note that for multi-unit 

companies, forms are checked-in at the establishment level and not the company level. Therefore, the check-in rates 

are based on the establishments within each of the companies selected. 

Table 2: Contact Exchange Form Check-in Rates 

Treatment 

Contact No Contact 

Exchange Exchange 

Response Form Form 

Mode (n=607,992) (n=17,964) Difference 

Surveyor 52.0% 51.4% 0.6% 

Mail 32.4% 32.7% -0.3%

Overall 83.6% 83.6% 0.0% 

V. Targeted Advanced Mailing 

An advance letter was mailed to a select number of single-unit companies in certain industries informing them the 

2012 Economic Census was coming. Certain industries were targeted because of their historically low response 

rate. This letter was mailed out three months prior to the initial census mailout. The industries selected were as 

follows: restaurants, hotels and motels, gas and convenience stores, bars, electronic stores, beauty salons, real estate, 

auto and body repair, home construction, and day care. The goal of the test was to determine if sending an advanced 

letter increased overall response to the 2012 Economic Census. 

To conduct this test, a systematic sample of 10,000 cases based on industry and annual payroll was selected from 

companies in the targeted industries that were to be mailed the economic census form. Half of those companies 

were selected to receive the letter with the other half being a control group for analysis. 
5 

Table 3 shows the check-in rates for each of the treatments. Even though the results are not final, the difference in 

all the check-in rates are not significant. 

Table 3: Targeted Advance Mailing Check-In Rates 

Treatment 

Response 

Mode 

Received 

Mailing 

(n=3,467) 

Did Not Receive 

Mailing 

(n=3,509) Difference 

Internet 29.5% 29.5% 0.0% 

Mail 47.6% 47.9% -0.3%

Overall 76.8% 77.1% -0.3% 

VI. Reminder Phone Call and Letter-Only First Follow-up

This test contained two different strategies for single-unit companies: the reminder phone call and the letter-only 

first follow-up. The first strategy was an automated phone call reminding the company that their 2012 Economic 

Census was due soon. The phone calls were made about two weeks prior to the official due date. For the second 

strategy, cases were mailed only a letter that also contained information to complete the census via the internet 

verses a second paper form. The budget was available to make 75,000 phone calls and send 75,000 letters. To 

conduct this test, the randomly selected companies were divided randomly among the four possible treatments: 

phone call and letter, phone call and form, no call and letter, and no call and form (the control group). The goal of 

this test was to determine if making the phone call and sending only a letter in the follow-up increased overall and 

internet response while getting responses earlier. 

One issue that was discovered during the planning phase was that company phone numbers were only available from 

a few sources from a representative sample: the 2007 Economic Census and the Medical Expenditure Panel Survey 

– Insurance Component (MEPS-IC). Because of that, it was decided that the sample would be selected in three 

parts. The first was a sample of 7,500 companies for whom the Census Bureau did not have a phone number, but 

had their phone number researched. A match sample (cases that had the same industry and similar annual payroll) 

of about the same number were then selected to act as a comparison group. Each sample was then randomly divided 

between the letter-only and form treatments. The second was all the cases where only the MEPS-IC phone numbers 

were available. The remaining sample was selected from the companies with a phone number from their 2007 

Economic Census response. All the cases from the MEPS-IC and the 2007 Economic Census respondent sample 

were randomly assigned to one of the four treatment groups. 

In total, 149,977 companies were selected to receive one of the four treatments. Table 4 shows the size of each 

sample and overall size. During the mailout of the first follow-up, only a small number of companies in select 

industries actually received the letter-only as a majority of the companies inadvertently also received the form a few 

days later. The industries that received the letter-only were wholesale trade and professional, science and technical 

services. 

Table 4: Reminder Phone Call and Letter-only First Follow-up Sample Sizes by Treatment 

Call and 

Letter 

No Call and 

Letter 

Call and 

Form 

No Call and 

Form Total 

No Phone Number 3,750 3,738 3,750 3,739 14,977 

MEPS 2,207 2,208 2,208 2,207 8,830 

2007 Respondent 31,543 31,543 31,542 31,542 126,170 

Total 37,500 37,489 37,500 37,488 149,977 

For analysis purposes, respondents were split into two groups. The first group contains only cases for which 

industries received only the letter. The second group contains the cases in the industries that inadvertently received 

the form. The check-in rates for each of the treatments can be found in Table 5 for the letter-only group and Table 6 

for the letter and form group. Methods are being researched to compare the rates so they can be compared for the 

different tests. Instead, a two by two (phone reminder X follow-up) ANOVA was calculated on the mean response 

time of each of the groups for the responding companies that were not included in the certified mailing test. 
6 

Table 5: Phone Reminder and Letter-only Follow-up Check-in Rates 

Treatment 

Response 

Mode 

Call and 

Letter 

(n=3,527) 

No Call and 

Letter 

(n=3,562) 

Call and 

Form 

(n=3,559) 

No Call 

and Form 

(n=3,471) 

Internet 41.3% 41.0% 36.8% 35.7% 

Mail 43.5% 42.5% 46.5% 47.2% 

Overall 84.3% 82.8% 82.9% 82.6% 

 

 

Table 6: Phone Reminder and Letter-only and Form Follow-up Check-in Rates 

Response 

Mode 

Call and 

Letter/Form 

(n=26,199) 

No Call and 

Letter/Form 

(n=25,974) 

Call and 

Form 

(n=25,767) 

No Call 

and Form 

(n=25,498) 

Internet 37.0% 36.7% 32.2% 31.6% 

Mail 47.9% 47.7% 51.4% 51.2% 

Overall 84.6% 84.1% 83.2% 82.4% 

Treatment 

The results show the model with only companies in industries that received the letter-only was not significant: F(3, 

11,277) = 2.42, MSE=5,896.69. Figure 1 contains the confidence intervals for each of the means for each of the 

treatment groups. Figure 1 shows that none of the individual treatments are significant. It is important to note that 

the sample sizes are much smaller for this group compared to the second group, therefore the confidence intervals 

are much wider. Additionally, there may be an industry effect, which is not being accounted for. 

The results of the second group containing industries that received the letter and the form as a follow-up show the 

model was significant: F(3, 92,984) = 84.48, MSE=186,377.5. Figure 2 contains the confidence intervals for each 

of the means for each of the treatment groups. Figure 2 shows that the means are significantly different for each of 

the groups. 

82

84

86

88

90

92

94

Phone Call/Letter Phone Call/Form No Phone Call/Letter No Phone Call/Form 

Mean Response Time in Days 

Figure 1: Mean Response Time for Companies in Industries that Received 

Letter-Only with 95% Confidence Limits 
7 

82

84

86

88

90

92

94

Phone Call/Letter and Form Phone Call/Form No Phone Call/Letter and Form No Phone Call/Form 

Mean Response Time in Days 

Figure 2: Mean Response Time for Companies in Industries that Received 

Letter and Inadvertent Form with 95% Confidence Limits 

VII.Certified Mailing 

A certified mailing is a mail package for which a person has to provide a signature to acknowledge that it was 

received. In the 2007 Economic Census, certified mailings were sent to single-units in select low responding 

industries during the third follow-up and that resulted in an increase in response. For the 2012 Economic Census, it 

was decided to test the certified mailings on single-units again, but in addition to testing the effectiveness, the timing 

was also tested. Each of the selected low responding industries had half of their companies receive the certified 

mailing in the second follow-up and the other half during the third follow-up. The goal of this test was to determine 

if there was a difference in the overall response and time to respond between the companies that received the 

certified mailing and those that did not in each of the follow-ups. 

The analysts selected 96 different industries that yielded 131,203 companies that were eligible for follow-up at the 

time of sampling. Half of the companies were randomly selected to receive the certified mailing in the second 

follow-up (65,604 companies total) with the other half (65,599 companies total) in the third follow-up. 

Figure 3 shows the overall check-in rate over time for each treatment. It shows that the certified mailings greatly 

increases the check-in rates for companies that received the certified mailing. More research is being done into 

whether the second certified mailing had more of an impact than the third certified mailing. 
8 

Figure 3: Certified Mailing 

VIII. Third Quarter Birth Letter-Only Follow-up

After it was realized that many of the letter-only first follow-up companies also inadvertently received a form in the 

first follow-up, it was decided to conduct a letter-only test on the third quarter birth single-unit companies. These 

were new companies that were not originally eligible for selection in the 2012 Economic Census but became eligible 

during the mailout period. The initial mailout for these companies happened in March 2013 with their first followup occurring in May 2013. The goal of the test was to determine if sending a letter only follow-up increased overall 

and internet response 

A letter-only follow-up was sent to a randomly selected group of 10,000 companies that were eligible for the followup. Table 10 shows the check-in rates for each of the treatments. The overall difference was significant when these 

results were calculated. The mail and internet check-in rates were also significant as the companies that received the 

letter-only were more likely to respond via the Internet. 

Table 10: Third Quarter Birth Check-in Rates 

Treatment 

Response 

Mode 

LetterOnly 

(n=9,491) 

Form 

(n=41,541) Difference 

Internet 31.8% 22.4% 8.4%* 

Mail 22.4% 29.9% -7.9%*

Overall 53.7% 52.0% 1.7%* 

* represents the difference was significant 
9 

IX. Conclusions and Next Steps 

Seven response and contact strategies (six original and one additional) were tested on companies in the 2012 

Economic Census in an attempt to increase overall response. Most of the results are not final and are still being 

monitored as the Census Bureau is still collecting the data for the Economic Census. The Industry Classification 

Form (refile) is the only completed test and that showed no significant difference in the overall check-in rate, but the 

companies that received the follow-up letter were more likely to respond on the Internet. There are possible 

significant differences in some of the other tests. The phone call reminder and letter-only and inadvertent form test 

currently shows that both tests affected the response time. The letter-only third quarter birth test currently shows the 

letter increased response, especially internet response. If the difference remains, the cause of the difference will be 

researched. The check-in rates will also be modeled for each of the tests to look at the effects. Differences will also 

be looked for at various descriptive categories (i.e. industry, size). Potential cost savings will be evaluated for each 

of the strategies using cost models and response rates, which will become part of management’s decisions in 

pursuing response improvement strategies for the future surveys and censuses. 

The Census Bureau is currently discussing the response and contract strategies for the 2017 Economic Census. 

Results of this research will become part of making decisions about which strategies to pursue. It is hoped that this 

research will also encourage the further use of experimental design in assessing improvements to survey data 

collection of companies. 

X. References 

Chestnut, John. 2010. “Testing an Additional Mailing Piece in the American Community Survey” Retrieved 

September 13, 2013, from 

http://www.census.gov/acs/www/Downloads/library/2010/2010_Chesnut_01.pdf. 

Millar, Morgan M., Allison C. O’Neill, and Don A. Dillman. 2009. “Are Mode Preferences Real?” Retrieved 

September 13, 2013, from 

http://sesrc.wsu.edu/dillman/papers/2009/Tech%20Report%20FINAL%20Feb%2023.pdf. 

Trancreto, Jennifer, Mary Frances Zelenak, Mary Davis, Michelle Ruiter, and Brenna Matthews. 2012. “2011 

American Community Survey Internet Tests: Results from First Test in April 2011” Retrieved September 

13, 2013, from http://www.census.gov/acs/www/Downloads/library/2012/2012_Tancreto_01.pdf.