Does giving people their preferred survey mode 

actually increase survey participation rates? 

An Experimental Comparison

Kristen Olson, Jolene Smyth and Heather Wood

Survey Research and Methodology Program

University of Nebraska-Lincoln

1

Paper presented at the annual meeting of the American Association of Public 

Opinion Research, Chicago, IL, May 13-16, 2010.
Department of Sociology and

Survey Research and Methodology Program

Mixed Mode Surveys and Mode Preference

• Mixed mode surveys are becoming increasingly 

common to counter falling response rates

• Long-standing hypothesis: 

– People prefer certain modes over others (e.g., Groves and Kahn 

1979) 

– Offering people a preferred mode will increase their 

probability of participation in that mode

– Still an open question

2
Department of Sociology and

Survey Research and Methodology Program

Mode preference is different 

from mode choice

• Mode preference

– Positive attitude about participating in one mode over 

other modes

– Can express this preference prior to participating in a 

particular survey

• Mode choice

– Behavior of selecting a mode to participate in when 

offered two or more mode choices

– Engages in this behavior during a particular survey

3
Department of Sociology and

Survey Research and Methodology Program

Research Questions

• Do people who express a preference for a particular 

mode participate at higher rates in that mode than 

others?

– Is this due to contactability or amenability?

• When given a choice, do people select the mode they 

reported preferring? 

– How does the order in which the modes are offered affect 

participation in a certain mode?

• Does mode preference affect the timing of participation 

in that mode?

4
Department of Sociology and

Survey Research and Methodology Program

Why might mode preference matter?

• Leverage-Saliency Theory (Groves, Singer and Corning 2000) and

Social Exchange Theory (e.g., Dillman, Smyth and Christian, 2009) 

– Both theories predict that offering a person the mode 

they prefer should increase cooperation rates

• Mode preference should be an indication of 

cooperation in a particular mode, not contactability

– Requires an interviewer-administered mode to 

disentangle the two sources of nonresponse

5
Department of Sociology and

Survey Research and Methodology Program

Measuring Mode Preference

• 2008 Nebraska Annual Social Indicators Survey 

(NASIS)

– Omnibus survey conducted by UNL’s Bureau of 

Sociological Research February – August 2008

– CATI survey with listed landline sample of Nebraska 

residents age 19+

• Estimated 23% of Nebraska households were cell-only in 2007 

(Blumberg et al. 2009)

• Response Rate (AAPOR RR3): 38%

– Respondents to the 2008 NASIS were asked if they would 

be willing to participate in future social research projects

• n = 1370 of 1811 (75.6%) said yes

6
Department of Sociology Department of Sociology and

Survey Research and Methodology Program

Mode Preference Question

• If you received a request to do 

another survey like this one, would 

you prefer to participate in...

– An in-person interview at your home

– An interview on your home phone

– An interview on your cell phone

– A paper survey sent by mail, or 

– A survey on the internet

Missing

4.5%

In 

person

1.7%

Home 

Phone

49.2%

Cell 

Phone

0.4%

Mail

24.6%

Internet

19.7%

7
Department of Sociology and

Survey Research and Methodology Program

2009 Quality of 

Life in a Changing 

Nebraska Survey

July – Oct. 2009

Mail

Web

Mail

N=297

RR2=53.9%

Web

N=314

RR2=50.0%

Mail

N=327

RR2=53.2%

Web

N=291

RR2=25.4%

8

Step 1. Measure 

Mode Preference

Step 2. Evaluate Participation

2009 NASIS

March – July 2009

2008 NASIS

Feb. – Aug. 2008

Phone

N=1000

RR2=55.5%

1370 Respondents 

were willing to 

participate in 

additional research
Department of Sociology and

Survey Research and Methodology Program

Analytic Methods

• Received preferred mode as first mode

– Prefer any ‘interviewer-administered survey’ and received 

the phone survey or

– Prefer ‘web’ and assigned to ‘Web Mail’ or ‘Web Only’ or

– Prefer ‘mail’ and assigned to ‘Mail Web’ or ‘Mail Only’

• Bivariate and multivariate analyses

– Item missing data multiply imputed (m=5) using sequential 

regression in IVEWare

– All analyses account for the clustering of requests within 

people

9
Department of Sociology and

Survey Research and Methodology Program

Research Questions

• Do people who prefer a particular mode participate 

at higher rates in that mode than others?

– Is this due to contactability or amenability?

• Do people select the mode they prefer? 

– How does the order in which the modes are offered affect 

participation in a certain mode?

• Does mode preference affect the timing of 

participation in that mode?

10
Department of Sociology and

Survey Research and Methodology Program

Mail 

Survey

Web

Survey

Phone 

Survey

In single-mode surveys, response rates are 

higher for those being surveyed in their 

preferred mode

57.7

52.7

40

21.6

60.3

49.3

0

10

20

30

40

50

60

70

80

90

100

Prefers 

Mail

Prefers 

Web/Intvwr

Prefers 

Web

Prefers 

Mail/Intvwr

Prefers 

Intvwr

Prefers 

Mail/Web

11

**

***
Department of Sociology and

Survey Research and Methodology Program

Mail 

Survey

Web 

Survey

Phone 

Survey

But the web survey response rates are lower than 

the mail and phone survey response rates across 

all preference groups

57.7 53.8 52.5

17.3

40

23.5

51.4

46.6

60.3

0

10

20

30

40

50

60

70

80

90

100

Mail Web Intvwr Mail Web Intvwr Mail Web Intvwr

Mode Preference

Mode Preference Matches Survey Mode

Mode Preference Doesn’t Match Survey Mode
Department of Sociology and

Survey Research and Methodology Program

Web Survey 

with Mail 

Supplement

Mail Survey 

with Web 

Supplement

In mixed-mode surveys, response rates are 

not higher for those who prefer the mode 

that is offered first

52.9 49.4 46.8

55.2

0

10

20

30

40

50

60

70

80

90

100

Prefers 

Mail

Prefers 

Web/Intvwr

Prefers 

Web

Prefers 

Mail/Intvwr

13
Department of Sociology Department of Sociology and

Survey Research and Methodology Program

Across all the modes, receiving the preferred 

mode increased response rates by 41% even 

when the following variables were controlled

• Mode (Mail, Web, Mail/Web Web/Mail)

• Mode * Received preferred mode

• Age

• Education

• Sex

• Has children

• Married

• Employment

• Income

• Has internet access

• Has cellular phone

• Lives in city

• Self-rated computer ability

• Self-rated hearing

• Self-rated health

• Concern about identity theft

• Depression

14

Based on logistic regression predicting overall survey participation in NASIS 2009 and 

QLCN with standard errors adjusted for multiple imputation and the clustering of 

requests within respondents. Red indicates p≤.05.
Department of Sociology and

Survey Research and Methodology Program

Cooperation rates are higher in the phone 

survey for those who prefer phone than 

those who prefer a different mode

76.2 79.2

71.5 69

0

10

20

30

40

50

60

70

80

90

100

Contact Rate Cooperation Rate

Contact or Cooperation Rate

Preferred phone

Did not prefer phone

15

* ***

When respondent characteristics 

are controlled.
Department of Sociology and

Survey Research and Methodology Program

Research Questions

• Do people who prefer a particular mode participate 

at higher rates in that mode than others?

– Is this due to contactability or amenability?

• When given a choice, do people select the mode 

they reported preferring? 

– How does the order in which the modes are offered affect 

participation in a certain mode?

• Does mode preference affect the timing of 

participation in that mode?

16
Department of Sociology and

Survey Research and Methodology Program

Web/Mail

Survey

Mail/Web

Survey

Mail

Survey

Web

Survey

People who prefer web choose to participate 

by web, but only when it is offered first

0%

20%

40%

60%

80%

100%

Web Mail Int Web Mail Int Web Mail Int Web Mail Int

Preferred Mode

Participated by Web Participated by Mail

17

* **

% of respondents who participated by web or mail, by mode preference
Department of Sociology and

Survey Research and Methodology Program

Web/Mail

Survey

Mail/Web

Survey

Mail

Survey

Web

Survey

The finding that people who prefer web choose to 

participate by web holds when analyses are limited to only 

those reporting internet access in the 2008 NASIS

0%

20%

40%

60%

80%

100%

Web Mail Int Web Mail Int Web Mail Int Web Mail Int

Preferred Mode

Participated by Web Participated by Mail

18

* *

% of respondents who participated by web or mail, by mode preference, 

for those reporting internet access in 2008 NASIS
Department of Sociology and

Survey Research and Methodology Program

Research Questions

• Do people who prefer a particular mode participate 

at higher rates in that mode than others?

– Is this due to contactability or amenability?

• Do people select the mode they prefer? 

– How does the order in which the modes are offered affect 

participation in a certain mode?

• Does mode preference affect the timing of 

participation in that mode?

19
Department of Sociology and

Survey Research and Methodology Program

Mail

Survey

Mail/ 

Web

Survey

Web/ 

Mail

Survey

Web

Survey

People who prefer web participate faster in the web 

survey than those who prefer other modes; little 

difference for mail preference, but...

40.9 40.3

35.3

41.4

26

18.8

28.3

15.2

0

10

20

30

40

50

60

70

80

90

100

Prefers 

Mail

Prefers 

Web/ 

Intvwr

Prefers 

Mail

Prefers 

Web/ 

Intvwr

Prefers 

Web

Prefers 

Mail/ 

Intvwr

Prefers 

Web

Prefers 

Mail/ 

Intvwr

Response Rates

20

Response rates before the reminder mailing 

(i.e., before the mode switch)
Department of Sociology and

Survey Research and Methodology Program

Mail

Survey

Mail/ 

Web

Survey

Web/ 

Mail

Survey

Web

Survey

Those who prefer a non-web mode participate at 

much higher rates when it is made available after 

a web mode

40.9 40.3

35.3

41.4

26

18.8

28.3

15.2

16.9

12.4 17.7 8

20.8

36.4

11.7

6.5

0

10

20

30

40

50

60

70

80

90

100

Prefers 

Mail

Prefers 

Web/ 

Intvwr

Prefers 

Mail

Prefers 

Web/ 

Intvwr

Prefers 

Web

Prefers 

Mail/ 

Intvwr

Prefers 

Web

Prefers 

Mail/ 

Intvwr

Response Rates

21

Response rates before and after the reminder mailing 

(i.e., before and after the mode switch)

**

*
Department of Sociology and

Survey Research and Methodology Program

Summary: Participation

• In single-mode surveys...

– People who preferred the web mode compared to those who preferred 

a different mode participated at higher rates in a web survey.

– People who preferred the phone mode compared to those who 

preferred a different mode participated at higher rates in a phone 

survey (due to cooperation).

– There was a slight, but insignificant tendency for people who preferred 

the mail mode to participate at higher rates in the mail survey

• BUT the advantage of catering to mode preference for web 

surveys may be lost to overall higher unit nonresponse rates 

in this mode.

– Those who preferred the web mode participated at higher rates in a 

web survey than those who preferred other modes, BUT they 

participated at lower rates in the web survey than they did in a mail 

and a phone survey.

22
Department of Sociology and

Survey Research and Methodology Program

Summary: Mode Choice

• When given mail first, most people responded by 

mail, regardless of mode preference.

• When given web first, over half of those who 

preferred web responded in this mode.

– Remaining respondents who preferred web and 

respondents who preferred mail responded subsequently 

by mail.

• Both of our designs were sequential mixed-mode 

designs. We did not test a simultaneous mixedmode design.

23
Department of Sociology and

Survey Research and Methodology Program

Summary: Timing

• People who preferred the web mode participated 

faster in the web survey than those who 

preferred other modes.

– Those who preferred other modes 

• Caught up when a subsequent mail option was provided

• Never caught up when web was the only mode offered

24
Department of Sociology and

Survey Research and Methodology Program

Implications

• Even though mode preference does lead to 

participation in that mode (when offered first, for 

some preferences), other considerations like overall 

response rates across different modes may be more 

important.

• We need to examine:

– How appealing to mode preference affects nonresponse 

bias. This work is getting underway.

– If speeding up response by catering to a web preference 

(with or without a subsequent mail mode) leads to a 

measurable reduction in costs without increasing 

nonresponse bias.

25
Department of Sociology and

Survey Research and Methodology Program

Thank you!

• Bureau of Sociological Research for data access

• Dan Hoyt for funding the initial data collection

• UNL Office of Research Layman Award for 

funding the second data collection

26