Incorporating A Multi-Mode Design Into A Random-Digit-Dialing Survey

Michael P. Battaglia1

, Larry Osborn1, Michael W. Link2, Martin R. Frankel3 and 

Ali H. Mokdad4 

1 Abt Associates Inc. 

55 Wheeler St., Cambridge, MA 02138. Mike_Battaglia@abtassoc.com, 

Larry_Osborn@abtassoc.com 

2 Nielsen Media Research 

3784 Ardsley Ct., Marietta, GA 30062. Michael.Link@Nielsen.com 

3 Baruch College, CUNY and Abt Associates Inc. 

14 Patricia Ln., Cos Cob, CT 06807. Martin_Frankel@abtassoc.com 

4 Centers for Disease Control and Prevention 

4770 Buford Highway NE, MS K-66, Atlanta, GA 30341, AMokdad@cdc.gov 

The findings and conclusions in this paper are those of the authors and do not necessarily represent 

the views of the Centers for Disease Control and Prevention, Abt Associates, Baruch College, 

CUNY or Nielsen Media Research. 

Abstract

Random-digit-dialing (RDD) surveys are typically conducted using only the telephone survey 

mode of data collection. However, because RDD survey response rates have been declining over 

the past ten years it is important to examine alternatives to the single-mode approach. This paper 

describes a test of one multi-mode alternative conducted as part of the Behavioral Risk Factor 

Surveillance System (BRFSS), a monthly state-based RDD survey that measures health conditions 

and risk factors. In six states, a list-assisted RDD sample was selected and matched with a 

database of residential telephone numbers and addresses. For the sampled telephone numbers with 

an address match, a mail survey with telephone survey follow-up of mail nonrespondents was 

employed. For sampled telephone numbers with no address match, the survey was conducted by 

telephone alone. After discussing the design and implementation of the six-state pilot survey, 

including procedures for the selection of the adult respondent(s) from the sample households, we 

focus on response rates and respondent characteristics by mode using the ongoing, single-mode 

BRFSS survey in each state as a comparison sample. The paper also discusses weighting and 

estimation issues for multi-mode RDD designs and examines potential mode effects. 

Key words: Behavioral Risk Factor Surveillance System, Mail Survey, Telephone Survey, 

Response Rates, and Mode Effects 

Introduction

The Behavioral Risk Factor Surveillance System (BRFSS), one of the world’s largest randomdigit-dialing (RDD) telephone surveys, tracks the health practices and risk behaviors of adults in 
the United States. State health departments conduct the BRFSS via continuous monthly telephone 

interviews using a standardized questionnaire, with technical and methodological assistance 

provided by the Centers for Disease Control and Prevention. BRFSS data are used to identify 

emerging health problems; establish and track health objectives; develop, implement, and evaluate 

a broad array of disease prevention activities; and support health-related legislative efforts. 

RDD telephone survey response rates have been declining over the past several years (Battaglia et 

al. 2007), as there is increasing use of call-screening devices coupled with increasing mistrust of 

unknown callers. In addition, there are growing threats to the validity of RDD sampling frames 

from cell phone–only households and telephone number portability. As part of efforts to explore 

alternative data collection methodologies for the BRFSS, and building on results from the BRFSS 

Mail Pilot Survey conducted in 2005 (Link et al. 2006), we designed and implemented the 2006 

Mixed Mode Pilot Survey (MMPS). The MMPS was designed to assess the effect of multiple data 

collection modes—specifically a mail survey with telephone follow-up—on BRFSS response rates, 

by using state-based sampling frames that accurately represent the adult population. Two sampling 

frames were selected for comparison. The first was a standard landline RDD frame. The second 

was an address-based frame: the U.S. Post Office’s Delivery Sequence File, which contains all 

addresses serviced by the U.S. Postal Service. Results from the MMPS are intended to provide 

information about using multiple data collection modes within alternative sampling frames for 

conducting cost-effective, reliable surveys of the general public, thus helping to guide future 

decisions about the implementation of the BRFSS. This article focuses on the RDD multi-mode 

pilot survey. 

Background On Multi-Mode Survey Designs

Multi-mode surveys combine different ways of collecting data. The modes of data collection 

include in-person, telephone, mail, Web, etc. In terms of cross-sectional surveys, there are three 

primary types of multi-mode survey designs. In the first, often referred to as “mode assignment,” a 

sample can be divided into subsets defined in some manner, and different modes can be applied to 

the subsets. In the research presented here, we used a mode assignment design. A second multimode design is “sequential.” The American Community Survey is the largest example of a 

sequential multi-mode survey. The first phase of data collection in such a design uses a mail 

survey. Households that do not respond to the mail mode are contacted via telephone at the second 

phase of data collection. Households that do not respond to the telephone survey are subsampled, 

and the subsample is sent to field interviewers who attempt an in-person interview at the third 

phase of data collection. The third type of multi-mode design is a “concurrent” mode approach. 

This involves offering the respondent multiple channels at once for completing the survey (e.g., 

telephone, Web). The respondent then chooses the preferred mode of responding. 

In deciding to use a multi-mode design instead of a more traditional single-mode design, one must 

assess the potential costs and benefits. Within a given survey budget, the goal of using a multimode approach is to decrease the total survey error of the key estimates. Multi-mode designs may 

raise response rates, but compared with a single-mode design, the increase may not occur across 

the entire sample. Some multi-mode designs can potentially shorten the time frame for data 

collection; however, a sequential approach could substantially lengthen the field period. A multimode design can reduce survey costs, provided that less costly modes are used before using the 
more expensive modes. The use of multiple modes may raise issues of comparability across 

modes. For example, questions asked by a person, as opposed to on paper, may be more likely to 

invoke socially desirable responses (Turner et al. 1998). Questions asked on paper are more likely 

to ensure privacy and allow the respondent to complete the survey at his or her convenience. 

However, complex forms in which some questions are to be skipped cannot be used, and literacy 

issues should be considered. Often in a multi-mode design, different modes are used for various 

subsets of the sample, making it difficult to determine if there is a mode effect. This can be 

overcome by building randomized experiments into the survey design. Another way around this 

issue is to take a “unimode design” approach, as suggested by Dillman (2000). The unimode 

approach may, however, limit the full potential of the modes used in a survey. 

The use of multi-mode survey designs has increased for cross-sectional surveys. Will they make 

single-mode designs a thing of the past? This is currently unknown, but it is clear that research 

needs to be conducted on multi-mode designs so that better design decisions can be made. The 

survey on which this article is based contributes to that discussion through its use of an RDD multimode design as part of a six-state pilot survey. 

Design And Implementation Of The RDD Multi-Mode Pilot Survey

Six states were selected for participation in the RDD multi-mode pilot survey: California, Florida, 

Massachusetts, Minnesota, South Carolina, and Texas. These states were selected for a number of 

reasons. First, the states typically have yearly BRFSS response rates below 50%. Second, they are 

representative of the major U.S. geographic regions. Finally, combined, the states provided a good 

representation of the racial and ethnic mix of the U.S. population. 

For the landline RDD sample for each state, we selected 10-digit telephone numbers by using the 

GENESYS Sampling System. Each sample was selected from 100 telephone number banks with at 

least one residential, directory-listed telephone number per bank, an RDD sampling methodology 

commonly referred to as the list-assisted design. A telephone number bank consists of 100 

consecutive numbers within working exchanges. The sample from each state was divided into two 

strata: 1) directory-listed, residential telephone numbers and 2) non–directory-listed, residential 

telephone numbers (sometimes referred to as unlisted numbers). For each state, a proportionate 

stratified sample design was used to select a sample of telephone numbers from each stratum. The 

sample was divided into 30 random subsamples called replicates. The GENESYS Sampling 

System’s CSS procedure identified and flagged any selected telephone numbers that were 

determined to be business numbers, nonworking numbers, or fax/modem lines. These numbers 

were not called by interviewers but remained a part of the overall sample. A total of 23,580 

telephone numbers (between 3,840 and 4,020 per pilot state) was drawn for the RDD sample. 

After flagging business numbers, nonworking numbers, and fax/modem lines, 12,187 remaining 

numbers were released for data collection. 

Sampled RDD telephone numbers were matched against the TargusInfo database in order to 

identify cases with a usable mailing address. An address match of 61.9% was obtained for the 

overall sample. 

The RDD MMPS mail questionnaire consisted of the same questions asked in the core section of 

the BRFSS telephone interview and covered the following topics: General Health, Health Practices, 
Demographic Information, Health Problems, Diet, Activity Level, HIV/AIDS, and Life 

Satisfaction. 

In many RDD studies, including the BRFSS, a single adult respondent is chosen at random from all 

the adults in a household for a detailed interview. However, very little is known about such 

random selection processes in surveys that are mailed, with the 2005 BRFSS Pilot Mail Survey 

being one of the few studies to test such procedures (Battaglia et al. 2006). Building on results of 

the earlier study, three techniques for within-household respondent selection were tested as part of 

the RDD MMPS. Sampled records were randomized to one of the following respondent-selection 

methods: 

 Version A—The household member 18 years of age or older with the next birthday was 

instructed to complete the questionnaire (a variant of an approach commonly used in RDD 

surveys); 

 Version B—The household member 18 years of age or older with the last birthday was 

instructed to complete the questionnaire (a variant of the Version A approach); and 

 Version C—Every household member 18 years of age or older was asked to complete the 

questionnaire. 

Version A and B households were sent one questionnaire and one pre-paid reply envelope. Version 

C households were sent three questionnaires and three pre-paid reply envelopes. Version C 

households with more than three adults were asked to call the project’s toll free telephone number 

to request additional materials. 

The telephone questionnaire used the standard within-household respondent-selection process, that 

is, random selection of one adult in the household based on an implicit household roster. 

RDD cases that did not have an address match were immediately released to a telephone center. 

Trained interviewers dialed these cases and conducted the interview over the telephone as 

applicable. The interviews followed standard procedures in place for ongoing BRFSS RDD data 

collection, including the random selection of one adult in the household as the subject of the 

interview. Dialing of these cases began on June 20, 2006, and was completed on August 22, 2006, 

by which point the cases had been sufficiently dialed. A total of 4,742 RDD sample cases without 

an address match were released. 

An initial questionnaire package was prepared for each address-matched case in the RDD sample. 

The packages were bundled and sent to the health department in each state. Health department 

staff then mailed out the packages immediately upon receipt. This process was used so that the 

state’s postmark would appear on the outer envelope. Using the state’s postmark was another way 

to reinforce the state’s sponsorship of the study. 

For mail survey cases, a series of follow-up contacts (postcard reminders and second questionnaire 

mailings) was made prior to releasing cases for telephone follow-up. Telephone follow-up for 

nonrespondents began approximately four weeks following the second questionnaire mailing. The 

purpose of telephone follow-up was to immediately conduct the interview over the telephone. A 

total of 4,720 RDD sample cases with an address match were released for telephone follow-up. 
In the RDD-based sample, 3,918 surveys were eventually completed (3,046 via mail and 872 via 

telephone) from 3,263 unique households. 

An RDD-specific interview file was created by retaining all completed RDD sample cases. Five 

cases in which the ages of the respondents were reported to be less than 18 years were dropped 

from the interview file. Cases were categorized depending on their address-matched status and 

how they had completed the interview: address-matched cases that completed by mail, addressmatched cases that completed by telephone, and non-matched cases that completed by telephone. 

As described earlier, cases that were completed by telephone involved random selection of one 

adult from the household to be the interviewee, while cases completed by mail were randomized to 

one of three within-household respondent-selection methods: next birthday, last birthday, or all 

adults. 

To facilitate the weight calculations, the following variables were imputed for cases in which no 

valid response was available: the number of adults in the household, the respondent’s age, the 

respondent’s Hispanic/Latino status, and the respondent’s education level. Two additional 

variables were imputed for the RDD weight calculations: the number of telephone lines in the 

household and interruption in telephone service of a week or longer in the past year. 

The base sampling weight (BSW) for a state equals the number of telephone numbers in the 

sampling frame for the state divided by the sample size of telephone numbers in the state. The total 

number of telephone numbers in the six states is 84,673,400. Dividing that total by six (the number 

of pilot states) equals 14,112,233.333. For each state, the values of BSW for the completed 

interviews were summed (Z). For the completed interviews in each state, BSW_EQUAL was 

calculated as BSW x (14,112,233.333/Z). The sum of the weights in each pilot state was the same 

such that no one state dominated the weighted estimates. 

Next, a design weight (BSW_NUM_PHONE) was calculated to adjust for the number of nonfax/modem only telephone lines in each sampled household. BSW_NUM_PHONE = BSW 

divided by the number of non-fax/modem only telephone lines in the household, with the 

maximum telephone lines capped at three. 

A nonresponse adjustment (BSW_2) was next made for the number of interviews completed in a 

household, calculated as BSW_2 = BSW_NUM_PHONE multiplied by the number of adults in the 

household, with the maximum number of adults capped at five. 

A further adjustment (BSW_3) was made to Version C cases completed by mail, since more than 

one questionnaire could be completed in those households. BSW_3 was calculated as BSW_2 

multiplied by the ratio: (number of adults in the household/number of adults in the household who 

completed a questionnaire), with the maximum value for number of adults in a household capped at 

five. For Version A– and B–completed questionnaires, and for Version C questionnaires 

completed by telephone, BSW_3 = BSW_2. 

A nonresponse adjustment was next made to account for address-matched cases having a higher 

response rate. Cases with an address match were initially approached by mail, with 

nonrespondents later contacted by telephone. Cases without an address match were approached 
only by telephone, resulting in a lower response rate for this group. Thus, a unit nonresponse rate 

was made in each pilot state. First, all sampled telephone numbers with an address match and a 

case disposition indicating known households were counted, by state (R+NR_address). Next, all 

sampled, address-matched numbers with a case disposition specifically indicating a completed 

interview were counted, by state (R_address). Afterward, a ratio was formed: RATIO_address = 

R+NR_address / R_address. Following the same procedures, but using only sampled telephone 

numbers without an address match, a second ratio was formed: RATIO_no_address = 

R+NR_no_address / R_no_address. In each state, for address-matched cases, BSW_3_NR = 

BSW_3 x RATIO_address. In each state, for cases without an address match, BSW_3_NR = 

BSW_3 x RATIO_no_address. 

For all completed questionnaires in a state combined, BSW_3_NR was raked to population control 

totals (provided by Claritas, Inc.) for twelve age-by-gender cells (males and females aged 18-35, 

25-34, 35-44, 45-54, 55-64, and 65+ years) to produce a poststratified weight (BSW_4). Though 

initially kept separate, males aged 18-24 were combined with males aged 25-34, and females aged 

18-24 were combined with females aged 25-34 because of the small sample size in the younger age 

groups. Race/ethnicity (white/non-Hispanic versus all other race/ethnic groups) was also included 

as a raking margin. The raking was run using the median weight plus five times the interquartile 

range of the weights as the maximum weight value. 

For the completed interviews in each state, BSW_4_EQUAL (an equalized BSW_4) was calculated 

as BSW_4 x (68,565,464/6/sum of BSW_4). 

A second raking was run in each of the pilot states, again using BSW_3_NR as the input weight. 

The margins included in this raking were age by gender and race/ethnicity (provided by Claritas, 

Inc. and using the categories described earlier), and education (from the 2006 March Current 

Population Survey Supplement and using four categories—less than High School; High School 

graduate; some college; college graduate). In addition, an interruption in telephone service margin 

was included (created using data from the 2006 March Current Population Survey Supplement and 

completed RDD interviews, and having interruption/no interruption values). For the raking 

convergence criterion, a difference in percent of 0.025 was used. We ran the raking using the 

median weight plus five times the interquartile range of the weights as the maximum weight value 

until no weights exceeded the threshold. The raking weight was BSW_5. 

For the completed interviews in each state, BSW_5_EQUAL was calculated as BSW_5 x 

(68,565,464/6/sum of BSW_5). The sum of the weights in each state was the same such that no one 

pilot state dominated the weighted estimates. BSW_5_EQUAL is used in the demographic and 

risk factor comparisons presented below. 

Response Rates

A key issue is whether the RDD multi-mode design approach resulted in a substantial increase in 

the response rate compared with a single-mode RDD survey. For comparison, we used the BRFSS 

survey conducted by the pilot states as the single-mode RDD survey, applying the AAPOR RR 4 

response rate formula to the BRFSS sample that was taken in the six states during the same time 

period as the RDD multi-mode survey (AAPOR 2006 – 4th edition available at www.aapor.org). It 
was also applied to the RDD multi-mode survey in the six states. We took the additional step of 

first matching the BRFSS RDD sample with a commercial database of residential addresses, 

allowing us to separate the BRFSS sample into two subsets: address-matched sample telephone 

numbers and sample telephone numbers without an address match. We calculated the response rate 

separately for each of the two subsets. For each subset, we determined the value of “e” (the 

estimated proportion of unresolved telephone numbers that are residential) by using the ratio of 

residential sample numbers to total resolved sample telephone numbers. 

For calculating the RDD multi-mode survey household-level response rates, the sample was first 

divided into two subsets: (1) sample numbers with an address match that were included in the mail 

survey mode of data collection and did not move onto the telephone mode of data collection (mail 

only cases) and sample numbers with an address match that were included in the mail survey mode 

of data collection and moved onto the telephone mode of data collection (mail/phone cases), and 

(2) sample numbers without an address match (phone-only cases). 

The response rate was calculated separately for each subset. For each subset the value of “e” was 

calculated from the resolved sample. Response rates were also calculated for the overall BRFSS 

sample and for the overall RDD multi-mode sample (Table 1). For all six states combined, the 

BRFSS response rate was 35.3% compared with 45.3% for the RDD multi-mode survey (+10.0 

percentage points). A higher response rate for the multi-mode survey was seen in each state except 

South Carolina. The response rate for the RDD multi-mode survey differed considerably by 

subset: 49.9% for the address-matched sample, and 21.7% for the no address–match phone-only 

sample. 

The use of the two modes of data collection has raised the response rate in the address-matched part 

of the sample, while having no impact on the part of the sample without an address match. We can 

judge the size of the increase by comparing the response rate for the address-matched sample with 

the response rate in the BRFSS for the address-matched sample: 49.9% versus 35.3% (+14.6%). 

We can see that the multi-mode approach led to a substantial increase in the response rate 

compared with a single-mode RDD approach, but the increase occurred only in the addressmatched part of the sample. Adults in address-matched household may differ in important ways 

from adults in households that are not in commercial address databases. It is, therefore, possible 

that in increasing the response rate, we have increased nonresponse bias rather than decreased it. 

We investigated this topic by focusing on the characteristics of the adults who completed the RDD 

multi-mode survey. 

Demographic Comparisons

We compared the weighted demographic and socioeconomic characteristics of the BRFSS 

interviewees with those of the adults who completed the RDD multi-mode survey. The RDD 

multi-mode estimates are based on BSW_5_EQUAL. The BRFSS survey followed its current 

standard weighting methodology. The BRFSS design weights are poststratified to Claritas 

population control totals based on age-by-gender cells. In some states, the poststratification is 

based on age-by-gender-by-race/ethnicity cells. The poststratified BRFSS weights were then ratio 

adjusted so that the weights in each of the six states summed to the same total. This “equalization” 

of the weights prevents a large state, such as California, from dominating the estimates. We also 

compared the RDD multi-mode demographic and socioeconomic characteristics with 
Claritas/March CPS control totals. We also “equalized” the CPS weights so that the six sates could 

be combined for comparison purposes. 

The demographic and socioeconomic distributions of the RDD multi-mode survey and the BRFSS 

survey were very similar. The percent of never-married adults was lower in the multi-mode survey 

(17.7% versus 23.9%). Focusing on the address-matched sample, the demographic and 

socioeconomic distributions were also very similar. The RDD multi-mode survey still had a lower 

percent of never married adults than the BRFSS (17.7% versus 23.4%). The RDD multi-mode 

survey also had a lower percent of adults living by themselves (13.1% versus 19.3%). 

When we compared the RDD multi-mode demographic and socioeconomic characteristics with the 

Claritas/March CPS control totals, we found that, as expected, the age, gender, and education 

distributions were in agreement—they were controlled in the weighting. The distributions across 

the five minority race/ethnicity categories were all close except for the multiracial non-Hispanic 

category (3.3% versus 0.8%). This difference could be attributable in part to slight differences in 

how CPS and BRFSS collect race information. Number of children in the household and number 

of adults in the household were not controlled in the RDD multi-mode weighting. Compared with 

the CPS, we did see some underrepresentation of adults in households without children (56.5% 

versus 59.6%), but the underrepresentation of adults living by themselves was smaller (14.0% 

versus 15.7%). Never-married adults were also underrepresented in the RDD multi-mode survey 

(17.7% versus 25.6%). By controlling on education, age, gender, and race/ethnicity, we could see 

that the multi-mode survey compared favorably with the Claritas/March CPS control totals. 

Risk Factor And Health Condition Comparisons

We selected eight key risk factors and health conditions from the BRFSS questionnaire. 

Comparisons of the percentage of adults at risk/with the health condition were made between the 

RDD multi-mode survey and the BRFSS survey (Table 2). We found that the overall estimates for 

each of the two surveys were very similar, for example, the percent of adults with health care 

coverage (80.9% versus 81.9%). An exception was that the estimate for the binge drinking risk 

factor was noticeably higher in the RDD multi-mode survey (18.2% versus 15.1%). This may be 

attributable to the sensitive nature of the alcohol-use questions in the BRFSS interviewerconducted telephone survey. 

To gain a better understanding of the differences between the two surveys, we also compared risk 

factors for the address-matched subsets of the BRFSS and RDD multi-mode samples. We found 

that all of the estimates were similar except for the binge drinking risk factor (18.2% versus 

15.2%). Other than that variable, there was no evidence of mode effects for the address-matched 

sample. 

Another “face validity” comparison is to use the National Health Interview Survey (NHIS), which 

collects data on some of the same risk factors as the BRFSS. The NHIS is conducted in person and 

achieves a response rate considerably higher than that of the RDD multi-mode survey. A key 

limitation of using the NHIS for this purpose is that we are comparing estimates for the six pilot 

states with national estimates from the NHIS. Even with this limitation, the risk factor estimates 

from the two studies were close. 
Our overall conclusion is that the multi-mode approach yields estimates that are reasonably close to 

the single-mode BRFSS estimates, and the differences are not large enough to suggest a change in 

key health care policies or prevention/education programs. Likewise, the comparison with the 

NHIS yields a similar conclusion. 

Cost Comparison

Cost is an important component in the evaluation of any survey design. The data collection costs 

per 1,000 completed interviews were calculated for both the RDD telephone survey and RDD 

multi-mode survey, using (1) actual unit costs for materials and supplies based on the pilot survey 

experience, (2) production statistics from the pilot survey, and (3) estimates of industry averages 

for direct hourly rates and indirect cost rates (i.e., fringe benefits, general and administrative 

expenses, indirect technical costs, and materials support expenses). Other costs assumed to be 

nearly equivalent regardless of the survey design were not included, such as overall project 

management, survey design development, and post-data collection weighting and analysis. 

We compared costs for the RDD single-mode and RDD multi-mode approaches assuming 1,000 

interviews. The cost for the RDD multi-mode approach was only about 2.5% less than that of the 

RDD single-mode survey to obtain the same number of completed interviews ($59,922 versus 

$61,438). The small cost difference is due to the substantially higher material and supply costs for 

the RDD multi-mode survey. 

Conclusions

The introduction of a mail survey component into a random-digit-dialing telephone survey offers 

several potential benefits: 

 Respondents have more flexibility in completing the questionnaire compared with an 

interview-based survey; 

 Materials that accompany the questionnaire mailing can be used to legitimize the survey; 

 Higher reporting of some sensitive items can be expected compared with an interviewerbased survey; 

 Different follow-up methods can be used (e.g., second questionnaire mailing); 

 Incentives can be delivered more easily; 

 There is a potential for lower costs. 

These benefits come with some limitations: 

 Within-household respondent selection is more difficult and error prone; 

 Complex question-skip patterns and long questionnaires cannot be used; 

 There may be literacy issues; 

 A longer field period is required compared with a single-mode survey; 

 The issue of “cellular phone–only” adults and adults living in households without 

telephones is not addressed. 
The RDD multi-mode survey demonstrates that it is possible to raise response rates by at least ten 

percentage points. Thus, if one’s objective is to achieve a response rate of at least 50%, the multimode design approach should be considered. A response rate at that level may be needed for “face 

validity” purposes or may be required by a journal for publication acceptance. But does raising the 

response rate reduce total survey error? For the RDD multi-mode survey, we only raised the 

response rate in the address-matched part of the sample. Adults living in households that are not in 

commercial address databases have different demographic and socioeconomic characteristics than 

address-matched adults. Thus, one is raising the response rate in one part of the sample relative to 

a part that has different characteristics. This differential could, therefore, actually have the opposite 

effect of increasing total survey error. At a minimum, one should strongly consider making a 

separate interview nonresponse adjustment for the address-matched sample versus the non– 

address-matched sample in the weighting procedures. 

Improving participation among subgroups of the sample that are underrepresented is perhaps a 

better yardstick for comparing alternative designs. Although the specific characteristics of the 

underrepresented groups can vary across populations of interest and survey designs, in most RDD 

surveys these groups tend to be younger adults (those 18 to 34 years of age), male, adults of lower 

education, and racial/ethnic minorities. Unfortunately, use of a multi-mode design did not appear to 

substantially improve responses among these particular subgroups. Finally, the cost difference 

between an RDD single-mode survey and an RDD multi-mode survey was relatively small. 

Acknowledgments

We would like to thank the following BRFSS Coordinators and Project Directors and their staff for 

their valuable assistance in the conduct of this pilot study: Nagi Salem (Minnesota), Norma McNab 

and Michelle Cook (Texas), Melissa Murray and Youjie Huang (Florida). Zi Zhang 

(Massachusetts), Beili Dong and Jennifer Chiprich (South Carolina), and Marta Induni (California). 

References

American Association for Public Opinion Research (AAPOR). 2006. Standard Definitions: Final

Dispositions of Case Codes and Outcome Rates for Surveys, 4th ed. Ann Arbor, MI: AAPOR. 

Battaglia, M., M. Link, M. Frankel, L. Osborn, and A. Mokdad. 2007. “An Evaluation of 

Respondent Selection Methods for Household Mail Surveys.” Public Opinion Quarterly

(forthcoming). 

Battaglia, M., M. Khare, M. Frankel, M.C. Murray, P. Buckley, and S. Peritz. 2007. “Response 

Rates: How Have They Changed and Where Are They Headed?” Advances in Telephone Survey

Methodology, Edited by J. Lepkowski, C. Tucker, M. Brick, E. de Leeuw, L. Japec, P. Lavrakas, 

M. Link, and R. Sangster. New York: John Wiley & Sons, Inc. (forthcoming). 

Dillman, D. 2000. Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley 

& Sons, Inc. 
Link, M. M. Battaglia, M. Frankel, L. Osborn, and A. Mokdad. 2007. “ Comparison of Addressbased Sampling versus Random-Digit Dialing for General Population Surveys.” Public Opinion

Quarterly (forthcoming). 

Turner, C., L Ku, and S. Rogers. 1998. “Adolescent sexual behavior, drug use, and violence: 

increased reporting with computer survey technology.” Science, 280:867-73. 
Table 1: Response Rates

BRFSS RDD MultiRDD: Mode:

No Address- No

Address- Address- Matched AddressMatched Matched Sample Matched

Overall Sample Sample Sample Overall Sample Overall Sample

Response Size1 Response Response Response Size1 Response Response

Rate Rate Rate Rate Rate Rate

Combined 35.3% 67,976 35.3% 32.3% 45.3% 12,187 49.9% 21.7% 

States 

using 

"equalized 

design 

weights" 

1 Genesys CSS purged telephone numbers excluded. 

Table 2: Comparison of Risk Factors and Health Conditions

2005 BRFSS RDD MultiRisk Factor NHIS RDD: Mode:

(n = 31428 (n = 21743 (n = 3913 

interviews) interviews) interviews) 

AddressNo Matched, Address- Address- No

National Address- Address- Mail Matched, Matched Address-

(50 States Matched Matched Only Mail/Phone Sample Matched

and DC) 

Estimate 

Overall

Estimate

Sample

Estimate

Sample

Estimate

Overall

Estimate

Sample

Estimate

Sample

Estimate

Overall

Estimate

Sample

Estimate

(%) (%) (%) (%) (%) (%) (%) (%) (%) 

Health care N.A. 81.9 82.6 76.0 80.9 85.0 74.7 81.7 75.9 

coverage 

Asthma 10.7 12.4 12.1 15.5 13.3 13.5 12.2 13.1 14.5 

Diabetes 7.4 9.3 9.9 8.5 10.5 11.1 10.6 11.0 7.0 

Cardiovascular N.A. 8.3 8.5 6.5 8.7 9.9 8.7 9.5 3.8* 

disease 

Obese (BMI > 23.2 22.9 22.7 24.9 24.7 24.3 26.0 24.8 24.0 

30) 

Current 20.7 20.1 19.8 22.7 20.3 17.0 24.5 19.4 26.2 

smoker 

Binge drinking N.A. 15.1 15.2 14.3 18.2* 17.9 18.8 18.2* 18.2 

Tested for 35.3 36.7 36.0 42.9 37.6 32.7 42.6 36.0 47.8 

HIV (under 

age 65) 

Note: * designates a difference between the RDD multi-mode survey and the BRFSS survey that is statistically 

significant at the individual 0.05 level.