Use of Text Messaging to Increase Response Rates 

Piper DuBray 

ICF International 126 College St. Burlington, VT 05401 

Proceedings of the 2013 Federal Committee on Statistical Methodology (FCSM) Research Conference 

Introduction 

Survey response rates have greatly declined in the past decade, causing researchers to seek new ways to increase 

participation. The Connecticut Department of Health (CT DPH) and ICF International conducted two pilot studies in 

2012 using text messages to 1) Increase response rates to the Behavioral Risk Factor Surveillance System (BRFSS) 

Cell phone Survey, and 2) Increase participation in the BRFSS Non-Response Web Follow-up Survey. 

The Centers for Disease Control and Prevention (CDC) established the state-level BRFSS in 1984. By 1994, all 

states and US territories are participating. In order to increase response to the landline survey, many states have 

adopted the use of advance letters to notify landline respondents whose phone numbers have been matched to an 

address of the upcoming BRFSS call. Cell Phone numbers are unlisted and therefore cannot be matched to 

addresses. As a result, cell phone respondents do not receive advance letters, nor do they receive the mailed 

invitation to participate in the BRFSS Web Survey. 

Research Questions 

The purpose of our research was to explore the effects of sending text messages to potential cell telephone and web 

survey participants. We wanted to determine whether sending a text message to cell phone sample in advance of 

upcoming calls increases response to the cell phone survey; also, whether sending a text message to cell phone nonresponders inviting them to complete the Web Survey increases web survey participation. Finally, we also examined 

whether the respondents who participate after receiving a text message any different from other respondents. 

Methods – Using Text Messages as Advance Notification to Cell Survey 

For our first experiment, we looked at sending text messages as advance notification for a cellular telephone 

interview. In this experiment, we divided cell phone sample from October 2012-March 2013 into 3 groups. Group 1 

was sent a text notification that included an offer of a ten dollar incentive, provided in cash by mail or an immediate 

Amazon.com gift code after completing the telephone interview. Group 2 was sent a text notification without the 

offer of an incentive. Group 3 was the control group and did not receive any text notification. 

The text messages were sent shortly after sample was received from CDC each month, prior to any attempts being 

made by the CATI center. Approximately 24 hours after the text was sent, ICF reviewed responses and removed any 

upset respondents from the phone study as a refusal.

The CATI program was modified to add questions to the end of the interview about the text message and to collect 

information for incentives. Questions added to CATI survey for Groups 1 and 2: 

1. Prior to our call today, we sent a text message to potential cell phone survey participants. Did you receive a text 

message about this interview? 

 1 Yes

 2 No

 7 Don’t Know 

 9 Refused 
If Group = 1 (incentive) and Q1 = 1 (recall text), Q2 was asked. 

Q2. In appreciation for the time you have spent answering our questions, we would like to provide you with $10 in 

compensation. Would you like to give us your address so that we can mail you the payment, or would you rather I 

give you a $10 Amazon.com gift code now? 

 01 Agree to give address information 

 02 Amazon.com gift code 

 99 Refused $10

Results - Using Text Messages as Advance Notification to Cell Survey 

In this first experiment, a total of 35,460 text message notifications were sent. Of the messages sent, 73 respondents 

(0.2%) replied to the message to opt out of the survey. Table 1: Sample Size shows the sample size for each pilot 

group. 

Pilot Group Sample 

1. Advance text with Incentive 11,492 

2. Advance text without Incentive 11,666 

3. No advance text 12,302 

Table 1: Sample Size 

In response to the question in the CATI survey, the method of incentive preferred was the gift code. The results are 

shown in Table 2: Preferred Incentive Mode. 

Incentive Mode Percent 

Preferred Cash Incentive 44%

Preferred Gift Code 53%

Refused Incentive 3%

Table 2: Preferred Incentive Mode 

The portion of the sample that was sent a text notification including an offer of an incentive, or Pilot Group 1, had 

the highest response and cooperation rates; however none of the differences were significant. Table 3: Response 

Rates shows the BRFSS Cooperation rate, Refusal Rate, and CASRO Response Rate for each pilot group.

Pilot Group Cooperation Rate Refusal Rate CASRO Rate

1. Advance text with 

Incentive 63.0% 9.1% 23.4% 

2. Advance text without 

Incentive 61.8% 8.6% 23.0% 

3. No advance text 59.4% 10.4% 22.5% 

Table 3: Response Rates 

Next, we compared demographic data between respondents in each pilot group. Income stood out as a significant 

(chi-sq=7.16, p=.0075) association with the text including incentive group (Group 1). Sending the notification with 

the incentive provided an increased response in the highest income category, while the lower income categories 

were more likely to have a higher response from the no-text group (Group 3). In the highest income category, 9% 
more respondents were from the group that was sent the notification including the incentive, than group 3, the no 

text group. In the income $35,000 < $50,000 category, 8% more respondents were from the no text group (Group 3) 

than the text with incentive group (group 1).In the income $25,000 < $35,000 category, 6% more respondents were 

from the no text group (Group 3) than from the text with incentive group (Group 1). These results are shown in 

Figure 1. 

0

5

10

15

20

25

30

35

40

< $10k < $15k < $20k < $25k < $35k < $50k < $75k $75k or

more

Percent Respondents in Pilot Group 

Income Category 

Text

With

Incentive

Text

Without

Incentive

No Text

Figure 1 Income Category by Pilot Group

During the BRFSS CATI interview, we asked respondents if they recalled receiving the text message. Of the two 

pilot groups that were sent text notifications, the recall of the text message was about equal. Overall, a third of the 

respondents recalled receiving the text message, however these differences were not significant, (P=.7377). It is still 

unclear from these results if those not recalling the notification simply do not remember it, or if they did not receive 

the message. 

Text Group

Recall Getting 

the Text 

Message

Did Not Recall 

Getting Text 

Message

DK if Received 

Text Message Refused

Advance text with 

Incentive 35% 55% 9% 1%

Advance text 

without Incentive 31% 58% 10% 1%

Total 33% 56% 10% 1%

Table 4: Text Recall

Methods – Using Text Messages as an Invitation to Web Survey 

In our second experiment, we reviewed the use of text messages as invitations to participate in an online survey. In 

2012, CDC conducted a Web Pilot for the BRFSS Survey, in which landline survey non-responders received a letter 

inviting them to complete the Web survey. During August and September of 2012, ICF conducted a pilot in which 

we sent a text message to non-responders of the CT BRFSS Cell Phone Survey, inviting them to complete the Webbased BRFSS survey. All text message invitations also included an option to opt out of future messages. 
Approximately one week after the first text message was sent, a reminder text was sent, excluding any respondents 

who opted out. 

The text message invitations were divided into two pilot groups. Group 1 was sent a text invitation that offered a $10 

Amazon.com gift code for completing the survey. 

Text 1 (160 characters): Participate in the CT Dept. of Public Health & CDC Study, receive $10 gift code 

https://www.healthdepartmentsurvey.org/ Login ID CT8xxxxx Reply STOP to opt out. 

Text 2 (132 characters): As a thank you, we will send you a text containing a $10 Amazon.com gift code 

approximately 4-8 weeks after you complete the survey. 

Group 2 received the same text invitation, however with no incentive offer. 

No Incentive Group (136 characters): Participate in the CT Dept. of Public Health & CDC Study 

https://www.healthdepartmentsurvey.org/ Login ID CT7xxxxx Reply STOP to opt out 

One week after the first invitation was sent, the same text was sent as a reminder to all who had not opted out. 

Incentive delivery text: Thank you for participating in the CT Dept. of Health and CDC study. Your $10

Amazon.com gift code is: xxx-xxxxxx-xxxx. 

Results - Using Text Messages as an Invitation to Web Survey 

During the two month experiment, a total of 4,847 text invitations were sent to BRFSS cell phone non-responders. 

Nine percent, or 425 respondents, replied “Stop” to opt out of the web invitation. Only 36 respondents participated 

in the web survey after being sent the text invitation. Table 5 shows the number of invitations sent in each group and 

the percent of those which completed the web survey.

Text Group Text Invitations Sent Web Surveys Completed Percent Completed

Group 1: With Incentive 2,424 26 1.1% 

Group 2: Without Incentive 2,423 10 0.4% 

Total 4,847 36 0.7% 

Table 5: Web Survey Participation after Text Invitation. 

The text message invitations produced a much lower response than the mail invitations that are sent to the landline 

BRFSS non- responders. The invitations that were mailed to the landline sample yielded 263 completed web 

surveys, a completion rate of 8.2%. 

Mail Invitations Sent Web Surveys Completed Percent Completed

3,208 263 8.2% 

Table 6: Web survey participation after mailed invitation.

Demographic data was then compared between web respondents that were recruited from the landline nonresponders by mail, and those recruited from the cell phone non-responders via text messages. There were two 

demographic areas, marital status and age, that stood out as having significant associations between the mail and text 

recruitment modes. More non-married completes, 31%, came from the group recruited by text with incentive, then 

either the text without incentive or mailing groups. However, only the mail vs. both text recruitment groups 
combined had a significant difference (P=0.0091). The two incentive groups within the text pilot group did not show 

significance (P=0.1121) however this is likely due to having such a small sample size in the text group.

The group recruited from the cell phone non-responders via text-message had 38% more 18-34 year olds as 

compared to the group recruited by mail. (P<0.001) Figure 2 shows the % of respondents in the web survey that 

were married and the % that were age 18-34, by mode of recruitment. 

Figure 2: Web Survey Demographics by Land and Cell Recruitment Mode 

67%

9%

44% 47%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Married Age 18-34

Land

Cell

Our last comparison looked at the demographic characteristics of respondents from the cell phone sample, between 

those who completed the cell telephone survey, and those who completed the survey via web. Education was the 

only category showing a significant difference between modes of completion. Within the cell sample respondents 

who completed the survey on the web, 85% had some college or higher level education; only 66% of cell sample 

respondents that completed on the phone had some college or higher level education. 

Results – Costs of Using Text Messaging. 

Sending text message advance notifications to cell phone respondents increased the cell phone cost per complete by 

3.8%. This considers the time and effort of sending the text messages and effort used to remove the opt-out records 

from the cell phone survey sample. 

Sending text message invitations to the web survey to cell phone non-responders increases the cost for web survey 

preparation by 27.7% over the cost of sending a mailed invitation to landline sample. The cost considers a 2-month 

fielding for both the mail and text web follow up; the mailing consisting of an initial letter followed by a reminder 

postcard, batched and sent every two weeks; the text invite consisting of an initial text followed by a reminder 

text, batched and sent every two weeks. 

Conclusions - Advance Text Message and Cell Survey Response 

We saw some increases in response, but none were statistically significant. Text messages with and without 

incentives are an option for improving cell response, but alone may not provide a significant increase. It may be 

worthwhile to consider text message notification and incentives, combined with other efforts to increase cell 

response. There were some between group differences on income. Texts with and without incentives may be 

considered if specific income groups are being targeted. 
Conclusions – Text Invitations to Web Survey 

A small number of completes were obtained through the text invitation, resulting in a total increase in Web 

participation of 13.7%. Using text message invitations means participation from people who otherwise would not 

have been reached. The additional cost required to send the text messages is considerable compared to the low 

number of completes that were obtained. There was a between group difference on marital status. Invitations by text 

message with incentives may be used to increase response from unmarried respondents. Also, recruiting Web 

respondents via cell phone produced a higher number of respondents with at least some college education.

Limitations 

These experiments with using text messages taught us that some carriers can block our outgoing number without us 

knowing. Therefore, we can only assume the text messages that we are sending out are being delivered by each 

carrier. We attempted to measure this by using a question of recall on those who did respond to the survey, which 

showed a 33% recollection rate of the text message. This could mean that 2/3 of our outgoing texts were not 

delivered. 

In the second experiment that used text messages as an invitation to the Web survey, the survey itself was not 

created to be viewed on smartphones. We may have lost some participation from respondents that could not view 

the survey directly on their phone. 

Sending the advance text messages to cell phones had an effect on the CATI center schedule, as we had to delay the 

start of calling monthly sample while we sent text messages and gave 24 hours to remove any who replied “stop” to 

opt out of the survey. 

Piloting additional studies with more data collection time points and the possibility of a larger overall incentive 

would be helpful in determining how best to implement this type of data collection effort and retain respondents 

over time.