Address-Based Versus Random-Digit Dial Sampling: 

Comparison of Data Quality from BRFSS Mail and Telephone Surveys 

Michael W. Link1, Michael P. Battaglia2, Martin R. Frankel3, Larry Osborn2, Ali H. Mokdad1 

1 Centers for Disease Control and Prevention 

4770 Buford Highway NE,MS K-66, Atlanta, GA, 30341, MLink@cdc.gov, AMokdad@cdc.gov 

2 Abt Associates Inc. 

55 Wheeler St., Cambridge, MA 02138, Mike_Battaglia@abtassoc.com, Larry_Osborn@abtassoc.com 

3 Abt Associates Inc. and Baruch College, CUNY 

14 Patricia Lane, Cos Cob, CT 06807, Martin_Frankel@abtassoc.com 

The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the Centers 

for Disease Control and Prevention or Abt Associates Inc. 

Abstract

As part of an on-going effort to expand coverage for and participation in the Behavioral Risk Factor Surveillance System 

(BRFSS), a pilot study was conducted to determine if a mail survey conducted with a random sample of adults selected from 

an address-only sampling frame (the USPS Delivery Sequence File) could rival the quality of the data collected using more 

traditional random-digit dial (RDD) methods. As one of the largest, RDD-based health surveys, the BRFSS is an ongoing 

surveillance system designed to collect uniform, state-specific data on preventive health practices and risk behaviors that are 

linked to chronic diseases, injuries, and preventable infectious diseases in the adult population. Moving from a telephoneonly approach to one which utilizes alternative survey modes and sampling frames has the potential to improve response rates

and increase the validity and reliability of the estimates obtained. This paper reports the results of a pilot study conducted in 

six states. Item non response rates and estimates derived from mail and telephone versions of the BRFSS are compared across 

8 key health conditions and risk behaviors, including diabetes, high blood pressure, obesity, alcohol use, and HIV testing. 

While a considerable amount of research and refinement will be necessary before DSF-based survey designs can be 

developed which truly rival current RDD designs, the data produced appear of reasonable completeness and the estimates 

produced are largely equivalent to those produced using current telephone survey methods. 

Key words: probability sampling, mail surveys, telephone surveys, public health surveillance 

Introduction 

In response to increased concerns about growing survey nonresponse and eroding coverage of random-digit dial (RDD) 

telephone sampling frames, researchers are testing new modes (or combinations of modes) and alternative frames from which 

to sample individuals in order to develop good quality, efficient, cost-effective household survey designs. In this study, we 

examine survey responses from one such effort which utilizes an address-based frame to conduct mail surveys with a 

probability sample of adults in six states. Comparisons are made with those from an on-going telephone version of the survey 

which employs a more standard RDD design. 

For more than three decades, RDD telephone surveys have been the predominant method for conducting surveys of 

the general public. Alternative designs of comparable speed, efficiency, and cost have been scarce. More recently, however, 

the growth of database technology has allowed for the development and maintenance of large, computerized address 

databases, which may provide survey researchers with an inexpensive alternative to RDD for drawing household samples. 

One such database is the U.S. Postal Service (USPS) Delivery Sequence File (DSF), which contains all delivery point 

addresses serviced by the USPS, with the exception of general delivery (USPS 2005). On the file, each delivery point is a 

separate record that conforms to all USPS addressing standards. Initial evaluations of the DSF as a means of reducing the 

costs associated with enumeration of urban households in area probability surveys have proven promising (Iannacchione, 

Staab, and Redden 2003; Staab and Iannacchione 2003; O’Muircheartaigh, Eckman, and Weiss 2002). Pushing assessment of 

the DSF further, in a previous analysis we showed that the DSF appears to provide state-level household counts similar to 

those report by the U.S. Census (Link et al. 2005). Coverage, however, does appear to vary significantly across different 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

types of areas, with lower coverage in more rural and lower income areas. Despite these limitations, the DSF does appear to 

be a potentially viable sampling frame for household-based surveys. When used in conjunction with mail surveys, the DSF 

may provide survey researchers with the advantages of cost effective sampling designs as well as a means of reducing survey 

nonresponse and reaching cell phone-only households and households without telephones which are missed by RDD designs. 

As one of the world’s largest RDD telephone surveys, the Behavioral Risk Factor Surveillance System (BRFSS) 

collects uniform, state-specific data on preventive health practices and risk behaviors linked to morbidity and mortality 

among adults. Since the mid-1990s, however, median state-level response rates for the BRFSS have declined sharply (Link 

2003). In an effort to curtail this trend, BRFSS has conducted a series of pilot studies designed to ensure the quality and 

longevity of the system (Link and Mokdad 2005a,b,c). In 2005, BRFSS piloted the use of a mail survey using the USPS DSF 

as the sampling frame. The goal of the pilot was to determine if use of this alternative design could provide higher coverage 

and lower nonresponse levels and presumably higher quality data than the current RDD approach. The study presented here 

provides analysis of responses to the mail survey pilot and makes comparisons with the on-going telephone version of 

BRFSS. In particular, we seek to answer the following: 

• How does data quality, measured in terms of item nonresponse, vary across the two designs? 

• Do the two designs produce different responses to key health condition and risk behaviors indicators? 

• Do responses to key items differ depending on the survey features used to improve survey participation in the mail 

survey, such as follow-up mailings, use of surname, and within household selection techniques? 

• Can an address-based mail survey design reach households typically missed by RDD designs, in particular, cell

phone-only households and if so how might responses from respondents in such households differ from those in

households with traditional landlines? 

Methods 

Six states participated in the mail survey pilot: California, Illinois, New Jersey, North Carolina, Texas, and Washington. 

These states were selected because (1) five of the six (NC being the exception) typically have yearly BRFSS response rates 

below 50%, (2) they represent the various geographic regions of the U.S., and (3) combined they provide a good 

representation of the racial and ethnic mix of the U.S. population. 

Mail Pilot Sample Design

The sample design for the on-going BRFSS is based on samples of 10-digit telephone numbers drawn from specific 

geographic stratum, typically states or health districts within states. For the mail survey pilot, however, the DSF sample 

frame is based on residential housing unit addresses (commonly referred to as deliverable residential addresses). The frame 

includes city-style addresses, and PO boxes and covers single-unit, multi-unit, and other types of housing structures. To 

ensure as complete coverage as possible we also included units in the frame identified by the USPS as being seasonal or 

vacant units, as well as throwback units (i.e., housing units that do not want mail delivered to their house, rather mail is 

picked-up at the local post office) and drop point units (i.e, locations where mail is dropped off and the housing units 

associated with that drop point pick up their mail at that location, such as a general store in a rural area or a trailer park). 

Marketing Systems Group (a database and survey sample vendor) provided access to the DSF frame and drew the 

state specific samples. The frame was first stratified by county FIPS code within each of the six participating states. Then 

separate samples of 1,680 addresses per state were drawn as a systematic random sample for a total of 10,080 addresses. We 

verified that the sample was distributed well within the state by checking the frequencies of the zip codes and the sectional 

centers (first 3 digits of the zip code). 

Pilot Study Experiments to Improve Participation

Embedded within the mail survey pilot were several split sample experiments designed to test the effectiveness of various 

contacting and within household selection procedures. These included: 

• Inclusion of surname/family name on the mailing envelope -- We asked Marketing Systems Group and 

TARGUSinfo (another database vendor) to match the sampled addresses with any name(s) they could associate with 

the address. The combined (Marketing Systems Group and Targusinfo) surname match rates were relatively high, 

ranging from 77.8% in New Jersey to 66.4% in Texas. Cases with a surname match were randomized in an equal 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

fashion into one of two groups (i) addressed to “The <Surname> Household or Current <State> Resident” or (ii) 

“<State> Resident”. Cases where a surname could not be matched were addressed to “<State> Resident.” 

• Postcard reminder -- All cases were equally randomized into one of two groups: (i) receive a postcard one week 

after initial questionnaire mailing or (ii) not receive a postcard. 

• Second questionnaire mailing -- All cases were equally randomized into one of two groups: (i) nonrespondents after 4 weeks 

receive a second mailing including cover letter and questionnaire or (ii) nonrespondents do not receive a second mailing. 

• Alternative within household selection techniques -- A literature search turned-up very little related to selecting one 

adult at random from a household in a mail survey (Battaglia et al 2005). Therefore, several techniques for within 

household selection were developed and tested. Sampled records were randomized equally to one of three 

respondent selection methods – (i) any adult in the household could respond with the household deciding who 

responds (a nonprobability approach hypothesized to have the lowest associated respondent burden and potentially 

producing the lowest level of nonresponse), (ii) adult with the next birthday (based on selection procedures used 

widely in a number of RDD surveys), and (iii) every adult in the household is asked to respond. 

Data Collection

The mail survey replicated the 2005 BRFSS core questionnaire, which consists of 75 questions asked by all states without 

modification in order or wording. Although the instrument was initially designed as a computer-assisted telephone interview 

(CATI) survey, only minor wording changes were required to convert the questionnaire into use for the mail survey. Survey 

packets were mailed to the sampled addresses and included a cover letter and questionnaire booklet. The return address on 

the original packet envelop was that of the state health department; the address on the return envelopes was that of Abt 

Associates (the data collection contractor). Data collection ran from March 15 through May 15, 2005 for California, Illinois, 

North Carolina, Texas, and Washington and from April 1 through May 30, 2005 for New Jersey. The mail survey data 

collection procedures were approved by the institutional review boards (IRB) of Abt Associates and of the six states. 

Mail Survey Weighting 

First, household base sampling weights (BSW) were calculated by state. The household base sampling weight for a state 

equaled the DSF population count of residential addresses divided by the sample size. One could use BSW for combined 

state-weighted response rate calculations, however, the larger states (California and Texas) would largely determine the 

resulting rates. It was therefore decided to ratio-adjust BSW so that the sum of the weights for sample households in each 

state summed to the average of the total residential addresses across the six states (6,124,232 = 36,745,391/6). Next, a design 

weight (BSW_2) for version 2 (respondent selection = next birthday) completed questionnaires was calculated as BSW times 

the number of adults in the household, where the maximum value for number of adults in a household was capped at 5. For 

version 1 (respondent selection = any adult) and version 3 (respondent sel ection = all adults) completed questionnaires, 

BSW_2 = BSW. A version 3 (all adults) nonresponse adjustment was made (BSW_3) and calculated as BSW_2 times the 

ratio: (number of adults in the household/number of adults in household that completed a questionnaire), where the maximum 

value for number of adults in a household was capped at 5. For version 1 and 2 completed questionnaires, BSW_3 = 

BSW_2. For all completed questionnaires in a state combined, BSW_3 was post stratified to 2004 population control totals 

(provided by Claritas) for 13 age by gender cells to produce a post stratified weight (BSW_4). Males aged 18-24 were 

combined with males aged 25-34, because of the small sample size in the younger age group. Finally, BSW_ 4 was ratioadjusted to produce a final weight (FINALWT) such that the sum of the weights in each state equaled the average of the total 

adult population across the six states (11,581,692=69,490,153/6). FINALWT was used to produce the estimates presented in 

the analyses below because it gave each state and “equal” contribution to the combined state estimates (i.e., the estimates 

were not dominated by California and Texas). 

RDD Survey 

The mail pilot surveys were conducted in parallel with the ongoing, monthly RDD data collection facilitating comparison of 

results across the two designs. Telephone survey data for the months of March, April, and May 2005 were used in this 

analysis. These data were weighted to adjust for sampling designs, post-stratified using the same gender and age categories 

specified for the mail survey data, and ratio-adjusted so that the sum of the final weights in each state equaled the average of 

the adult population totals across the six states. More details on BRFSS design and methodology are available elsewhere 

(Mokdad et al. 2003) and at http://www.cdc.gov/brfss. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Analysis 

Dichotomous measures of chronic health conditions (asthma, diabetes, high blood pressure, and obesity) and risk behaviors 

(smoking, binge drinking, HIV testing, and HIV risk behaviors) were examined. Asthma, diabetes, and high blood pressure 

were assessed by asking, “Have you ever been told by a doctor, nurse, or other health professional that you have 

[condition]?” Body mass index (BMI) was based on self-reported height and weight. Respondents were classified as obese if 

their BMI was ≥ 30 kg/m2

. Respondents who said that they currently smoke every day or some days were classified as 

“current smokers.” Binge drinking was assessed by asking, “Considering all types of alcoholic beverages, how many times 

during the past 30 days did you have 5 or more drinks on an occasion?” The HIV testing indicator was worded as: “Have you 

ever been tested for HIV?” Finally, engagement in behaviors linked to the transmission of HIV was assessed using the 

following: 

“After reading the list below, please tell us if any of the situations apply to you. You do not need to tell us 

which one: 

• You have used intravenous drugs during the past year. 

• You have been treated for a sexually transmitted or venereal disease in the past year. 

• You have given or received money or drugs in exchange for sex in the past year. 

• You had anal sex without a condom in the past year 

Do any of these situations apply to you?” 

The analysis was conducted in four parts. First, we compared the level of item nonresponse (i.e., “don’t know” or 

“refused”) for these 8 indicators across mode. For the mail survey, “don’t know” was made an explicit option for the items 

asking about asthma, diabetes, high blood pressure, but were excluded for the questions used to determine current smoking, 

binge drinking, obesity, HIV testing, and engaging in behaviors closely linked to HIV transmission. If an item was left blank 

or unanswered on the mail questionnaire it was coded as a refusal, with the exception of questions where the skip logic 

indicated a response was not necessary. In the telephone version of BRFSS, “don’t know” and “refused” options are available 

for nearly every question, but interviewers are trained not to read these options and to select them only if the respondent 

volunteers these answers. Second, we examined the relationship between survey design and survey responses using two-way 

contingency tables and logistic regression to adjust for respondent characteristics. Third, we assessed the impact of survey 

features designed to improve survey participation (i.e., use of surname, follow-up mailings, and within household respondent 

selection techniques) on survey responses using logistic regression and adjusting for respondent characteristics. Finally, a 

similar approach was used to examine the extent to which cell phone-only households and households with both landline and 

cell phone access differ from households with landline access only. All analyses were run using SPSS Version 13.0 with 

Complex Samples module (SPSS Inc 2004). 

Findings 

A total of 3,010 completed mail survey were obtained from the six states. Using response rate formula #4 recommended by 

the American Association for Public Opinion Research (AAPOR), the mail survey response rates for the six states ranged 

from 30.5% for New Jersey to 44.9% in Washington (AAPOR 2004). More extensive analysis of response rates by survey 

design features are available elsewhere (Link et al. 2005b; Battaglia et al 2005). From March to May, 2005, a total of 17,592 

telephone surveys were completed across five of the participating states (Illinois being the exception) with response rates 

ranging from 33.8% in New Jersey to 56.0% in North Carolina. Unfortunately telephone survey data were not available from 

Illinois in time for inclusion in this analysis. Comparisons of telephone and mail survey results presented in the first two 

sections below (Tables 1 and 2) are restricted to the other five states. Results focusing only on the mail survey in the final two 

sections (Tables 3 and 4) include data from all 6 states. 

Item Nonresponse 

Missing data values for survey items can reduce overall data quality. If use of particular modes leads to an increase in the 

level of item nonresponse, then the estimates produced may be less precise because of the reduced sample size (Mason, 

Lesser, and Traugott 2002). The rates of item nonresponse between the telephone and mail surveys were significantly 

different for 6 of the 8 health conditions and risk behaviors (Table 1). The rate of item nonresponse among the telephone 

survey responses was less than 1% for current smoking (0.4%), asthma (0.2%), high blood pressure (0.2%), and diabetes 

(0.1%). While these rates among the mail survey responses were low they were much greater than those of the telephone 

interview: current smoking (1.9%), asthma (2.4%), high blood pressure (1.7%), and diabetes (0.9%). The highest rates of 

item nonresponse in both surveys were for obesity (telephone survey = 8.2%; mail survey = 3.0%) and ever being tested for 

HIV (telephone survey = 5.6%; mail survey = 2.8%). In both cases item nonres ponse was significantly higher for the 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

telephone survey compared to the mail survey. No significant differences in the rates of item nonresponse were seen for 

binge drinking and HIV risk behaviors. 

Prevalence Estimates by Mode 

Comparing the weighted prevalence estimates across the two modes we found no significant differences on 5 of the 8 items 

(Table 2). A higher percentage of respondents to the telephone survey were classified as obese compared to those in the mail 

survey (29.0% versus 22.4%). In contrast, the mail survey produced higher estimates than the telephone survey for binge 

drinking (20.4% versus 12.5%) and engaging in behaviors that increase the likelihood of HIV transmission (6.9% versus 

4.1%). These differences persisted even after adjusting the responses to account for respondents’ state of residence, sex, race, 

age, education, and having health care coverage using logistic regression. When comparing the mail to the telephone survey 

the odds of obtaining responses indicative of obesity decreased by 17%, but increased by 77% for binge drinking and 74% 

for being in situations that could increase the chances of HIV transmission. 

Effect of Survey Design 

Survey design features such as the use of surname on the outside of mailing envelops, follow-up contacts, and within 

household selection procedures have the potential for influencing prevalence estimates to the degree that these factors 

increase or decrease participation among individuals with characteristics or behaviors associated with the measures in 

question. Table 3 provides adjusted odds ratios across the eight items for four of the mail survey design features using data 

from all 6 states. The main effects logistic regression models were also adjusted to account for characteristics of the 

respondents. Significant differences were most frequently noted among groups where a surname was or was not used on the 

mailing envelop. For asthma and diabetes the odds of obtaining a “yes” response decreased significantly among cases where 

a surname was identified and used on the mailing label, compared to cases where no surname was identified. There were no 

significant differences on these two variables between the groups where a surname was matched to the address but not used 

and the group where no surname was identified. For the two HIV-related items, both groups where a surname was matched 

differed significantly from the group where no surname was identified. The effects were similar as well. The odds of a 

respondent saying they had ever been tested for HIV decreased by 25% among those where a name was matched but not used 

and by 27% for those where the name was matched to the address and used on the label. Likewise, the odds of a respondent 

saying they had engaged in one of the behaviors linked to HIV transmission decreased by 62% when a name was identified 

but not used and by 67% when the name was used. 

Fewer effects were noted across the other design features. The use of a follow-up postcard increased the odds by 

25% that a respondent would report being a current smoker. Conversely, among those who were sent a second questionnaire 

mailing the odds of a respondent indicating they had engaged in behaviors linked to HIV transmission decreased by 44%. 

There were no significant differences noted across groups where different within household selection techniques were used. 

Effect of Type of Telephone Access in Household 

The mail pilot was successful in reaching households with different types of telephone access. Among the 2,953 adults 

completing the mail survey in the 6 pilot states, 13.7% indicated they had only a landline telephone, 79.4% had both a 

landline and a cellular telephone, 6.0% used a cell phone-only, and 0.9% had no telephone of any type within the household. 

Excluding the small number of cases with no telephone access (n=27), we examined potential differences in health conditions 

and risk behaviors across the three remaining groups using logistic regression (Table 4). The type of telephone access within 

a household was more closely related to risk behaviors than health conditions. The odds of a respondent indicating they had 

engaged in one or more behaviors linked to HIV transmission increased by nearly 400% among those in cell phone-only 

households compared to those in households with a landline only. The odds of a respondent reporting that they had engaged 

in binge drinking over the past 30 days were significantly higher in both cell phone-only households (+90%) and households 

with both landline and cell phone access (+56%). Conversely, the odds of a respondent being a current smoker were 30% 

lower among those with both a landline and a cell phone in the household compared to those with a landline only. 

DISCUSSION 

With a few notable exceptions, the weighted survey responses from the DSF-based mail survey were similar to those 

obtained from the RDD telephone survey design. While the rate of missing data was significantly higher in the mail survey 

than the telephone survey for 4 of the 8 items examined, the actual rates themselves were relatively modest (below 3.5% for 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

all 8 items). Missing data were more of a problem in the RDD design where rates for two of the items (being tested for HIV 

and obesity) exceeded 5%. The most interesting finding in terms of item nonresponse is the substantial reduction in item 

nonresponse for obesity using the mail survey mode of administration. Even for the three questions (asthma, diabetes, and 

high blood pressure) where “don’t know” was made an explicit response option on the mail survey and, hence, might be 

expected to result in greater item nonresponse, the rate of missing data was still below 2.5% for each of these questions. This 

may be because these questions are relatively straightforward factual questions (as opposed to opinion questions) on issues of 

person health on which most adults would be knowledgeable. 

Once weighting and post-stratification adjustments were made, the telephone and mail surveys also produced similar 

prevalence estimates for 5 of the 8 health conditions and risk behaviors. The higher reports in the mail survey for binge 

drinking and engaging in behaviors linked to HIV transmission are in line with other studies which have shown a greater 

likelihood for respondents to give socially desirable responses or being more reluctant to report accurately about sensitive or 

stigmatized behaviors in interviewer-administered surveys than in self-administered surveys (Link and Mokdad 2005a; 

Turner, et al. 1998; Dillman, et al. 1996; Aquilino 1994). A previous comparison of telephone and mail BRFSS survey 

respondents did not, however, find a significant difference in binge drinking behavior across the two groups (Link and 

Mokdad 2005b). Those findings were based on a sample of households drawn using RDD methods and reverse matching the 

telephone numbers to addresses and excluding potential households where an address could not be matched. The surveys 

were also conducted in different states than those in which the current mail surveys were conducted. 

Differences between the mail and telephone survey results were also observed for obesity. The differences (in terms 

of adjusted odds ratios) were not of the same magnitude for obesity as they were for the binge drinking and HIV risk 

behaviors items, but the differences were statistically significant. Obesity is determined based on self reports of height and 

weight. If either measure was missing then the value for obesity was declared missing. Of the 8 items examined, obesity had 

the highest percentage of item missing data for both the telephone survey (8.2%). This may indicate a significant level of 

reluctance to provide the height and weight information necessary to determine obesity which could have played a role in 

determining the differences found here. 

Focusing only on the mail survey results, the use of a surname or family name on the mailing label appears to be the 

factor of most concern. Use of surname was associated with lower reports for 4 of the 8 items. For two of the questions 

(asthma and high blood pressure) the effect appears linked directly to use of the name on the label as the estimates for the 

group where a name was available but not used did not differ significantly from those of the group where no name was 

available. It may be that persons with these conditions were more reluctant to report these conditions or to return a survey 

knowing that researchers knew at least their family name. It is less clear, however, why this might be the case with these 

presumably less sensitive health issues than more sensitive risk behaviors such as smoking or binge drinking. 

The effect on the two HIV-related items of having a surname available is interesting given that there was a 

significant decrease in reporting for both name-matched groups regardless of whether the name was used on the mailing label 

or not. This would indicate that the real difference is between cases where a name could be matched to an address (regardless 

of whether it was used on the label or not) and cases where a name could not be identified. These two groups, even after 

adjusting for respondent characteristics appear to be behaviorally different in terms of HIV-related behaviors. This distinction 

between surname-matched and non-surname-matched cases may be analogous to the distinctions seen in RDD surveys 

between cases where telephone numbers can be matched to addresses and those where they cannot (Link and Mokdad 

2005c). Those who are less likely to appear in the databases of major database firms such as Marketing Systems Group and 

TARGUSinfo are behaviorally different in important respects from those who are identified on these databases. 

One of the more encouraging findings from this DSF-based research is the success of the approach in reaching cell 

phone-only households and to a smaller degree households without telephones. The percentage of adults who indicated they 

live in cell phone-only households was very similar in this survey (6.0%) compared to that reported for a national face-toface survey (5.5%) conducted during the last 6 months of 2004 (Blumberg, Luke, and Cynamon 2005). The percentage of 

adults in households with no telephone was some what smaller, however, in the DSF-based mail survey (0.9%) as compared 

to the face-to-face survey (2.4%). These are both households missed or excluded by traditional RDD surveys. The findings 

also indicate that on some items (binge drinking and HIV risk behaviors) persons in these households are more likely to 

engage in these behaviors than those in households traditionally reached by RDD surveys. 

The current study does have several limitations. First, the DSF frame does not provide universal coverage of all 

households, particularly in more rural and lower income areas (Link et al 2005). Second, in the absence of more direct 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

measures (such as patient records or medical tests) we cannot determine which design (RDD telephone survey or DSF-based 

mail survey) is the most accurate. We know only that differences exist in some of the measures obtained using these modes. 

Third, the study was conducted in six states, which may not be representative of either the nation or other populations. 

In conclusion, while a considerable amount of research and refinement will be necessary before DSF-based survey 

designs can be developed which truly rival current RDD designs, the findings provided here are promising. The data 

produced appear of reasonable completeness and the estimates produced are largely equivalent to those produced using 

current telephone survey methods. Where there are differences, such as in higher reports of binge drinking and behaviors 

which can increase the likelihood of HIV transmission, the differences are as we might expect with the self-administered 

mode producing higher estimates than the interviewer-administered mode. DSF-based approaches should, therefore, continue 

to be examined as possible future alternatives or complements (particularly if used in mixed-mode mail and telephone 

designs) to RDD surveys of the general population. 

Acknowledgements 

We thank the following BRFSS State Coordinators for their critical assistance in the design and execution of this study: 

Bonnie Davis (California), Kenneth O’Dowd (New Jersey), Ziya Gizlice (North Carolina), Jimmy Blanton (Texas), and 

Katrina Wynkoop Simmons (Washington). We also thank Pamela Giambo and Sowmya Rao of Abt Associates for their 

assistance with this project. 

References 

American Association for Public Opinion Research 2004. Standard Definitions: Final Dispositions of Case Codes and

Outcome Rates for Surveys, 3rd Edition. Ann Arbor, MI: AAPOR. 

Aquilino, W.S. 1994. Interview mode effects in surveys of drug and alcohol use: a field experiment. Public Opinion 

Quarterly, 58, 210-40. 

Battaglia, M, M Link, M Frankel, A Mokdad. 2005. “An Evaluation of Respondent Selection Methods for Household Mail 

Surveys.” Presented at the 2005 Joint Statistical Meetings, Survey Research Methods Section, Minneapolis, MN. 

Blumberg, S, Luke, J, and Cynamon, M. 2005. “The Prevalence and Impact of Wireless Substitution: Updated Data from the 

2004 National Health Interview Survey.” Presented at the 2005 American Association for Public Opinion Research Annual 

Conference, Miami Beach, FL. 

Dillman, D. 2000. Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley and Sons, Inc. 

Dillman, D., Sangster, R., Tanari, J., and Rockwood, T. 1996. Understanding differences in people’s answers to telephone 

and mail surveys. In Braverman, M.T. and Slater J.K. (eds.), New directions for evaluation series, 70 (advances in survey

research). San Francisco: Jossey-Bass. 

Iannacchione, V, Staab, J, and Redden, D. 2003. Evaluating the use of residential mailing addresses in a metropolitan 

household survey. Public Opinion Quarterly, 76:202-210. 

Link, M. 2003. “BRFSS Response Rates: Which Way Is Up?” Presented at the 2003 Behavioral Risk Factor Surveillance 

System Annual Conference, St Louis, MO. 

Link, M., and A. Mokdad. 2005a. “Use of Alternative Modes for Health Surveillance Surveys: Results from a 

Web/Mail/Telephone Experiment.” Epidemiology (forthcoming). 

Link, M., and A. Mokdad. 2005b. “Effects of Survey Mode on Self-Reports of Adult Alcohol Consumption: Comparison of 

Web, Mail, and Telephone.” Journal of Studies on Alcohol, 66(2): 239-245. 

Link, M., and AH. Mokdad. 2005c. “Advance Letters as a Means of Improving Respondent Cooperation in RDD Studies: A 

Multi-state Experiment.” Public Opinion Quarterly (forthcoming). 
 

 

 

 

 

 

 

 

 

 

Link, M., M. Battaglia, P. Giambo, M. Frankel, A. Mokdad, and S. Rao. 2005a. “Assessment of Address Frame 

Replacements for RDD Sampling Frames.” Paper presented at the Annual Meeting of the American Association for Public 

Opinion Research, Miami Beach, FL. 

Link, M., M. Battaglia, M. Frankel, L. Osborn, A. Mokdad, and S. Rao. 2005b. “Assessment of an Address-based Sampling 

Frame Alternative to RDD: BRFSS Mail Survey Pilot Results.” Presented at the 2005 Joint Statistical Meetings, Survey 

Research Methods Section, Minneapolis, MN. 

Mason, R., Lesser, V., and Traugott, M.W. 2002. “Effect of item nonresponse on nonresponse error and inference.” In 

Groves, R.M., Dillman, D.A., Eltinge, J.L., and Little, R.J.A. (eds.) Survey Nonresponse. New York, NY: John Wiley & 

Sons. 

Mokdad, A., D. Stroup, W. Giles. 2003. “Public Health Surveillance for Behavioral Risk Factors in a Changing Environment

Recommendations from the Behavioral Risk Factor Surveillance Team.” Morbidity and Mortality Recommendations and

Reports, 52(RR09), pp.1-12. 

O’Muircheartaigh, C, Eckman, S, and Weiss, C. 2003. Traditional and enhanced field listing for probability sampling. 

Proceedings of the American Statistical Association, Survey Methodology Section (CD-ROM), Alexandria, VA, pp.2563-

2567. 

SPSS Inc. 2004. SPSS Complex Samples 13.0. Chicago, IL: SPSS. 

Staab, JM, and Iannacchione, VG. 2004. Evaluating the use of residential mailing addresses in a national household survey. 

Proceedings of the American Statistical Association, Survey Methodology Section (CD-ROM), Alexandria, VA, pp.4028-

4033. 

Turner, C.F., Ku, L., and Rogers, S.M., 1998. Adolescent sexual behavior, drug use, and violence: increased reporting with 

computer survey technology. Science, 280, 867-873. 

United States Postal Service. 2005. Delivery Sequence File (accessed April 12, 2005). Available at: 

http://www.usps.com/ncsc/addressservices/addressqualityservices/deliverysequence.htm. 
 

 

 

 

 

 

Table 1. Percentage of item missing data by survey design 

RDD telephone survey DSF-based mail survey 

Don’t know 

Don’t know / refused 

/ refused / blank 

Health condition / risk factor n %

1 n % 

Asthma 17,591 0.2 2,447 2.4*** 

Diabetes 17,591 0.1 2,447 0.9*** 

High blood pressure 17,591 0.2 2,447 1.7*** 

Obese (BMI > 30) 17,591 8.2 2,447 3.0*** 

Current smoker 17,591 0.4 2,447 1.9*** 

Binge drinking 17,591 2.3 2,447 2.1 

Tested for HIV2 13,308 5.6 1,778 2.8*** 

HIV risk behaviors2 13,308 3.7 1,778 3.4 

RDD = random-digit dial; DSF = Delivery Sequence File 

Significance: * p<.05, ** p<.01, *** p < .001 

Note: includes data from CA, NC, NJ, TX and WA. 

1

 Percentages are unweighted. 

2

 Questions not asked of respondents age 65 years or older 
 

Table 2. Prevalence estimates and adjusted odds ratios for effects of survey design on self-reports of 

health conditions and risk factors 

Health condition / 

risk factor

Prevalence estimates 

RDD telephone survey DSF-based mail survey 

% % 

n (95% CI) n (95% CI) 

1 Adjusted odds ratios

RDD telephone survey DSF-based mail survey 

AOR 

AOR2

(95%CI) 

Asthma 17,551 12.2 2,388 14.2 1.00 1.18 

(11.6, 12.8) (11.9, 16.9) (0.94, 1.47) 

Diabetes 17,571 8.4 2,425 7.6 1.00 0.94 

(7.9, 8.9) (6.4, 9.0) (0.77, 1.16) 

High blood pressure 17,560 26.5 2,406 27.4 1.00 1.09 

(25.7, 27.3) (24.9, 30.0) (0.94, 1.27) 

Obese (BMI > 30) 17,591 29.0 2,418 22.4 1.00 0.83* 

(28.1, 29.8) (20.1, 25.0) (0.71, 0.97) 

Current smoker 17,518 18.6 2,401 17.0 1.00 1.04 

(17.8, 19.3) (14.5, 19.9) (0.86, 1.28) 

Binge drinking 17,179 12.5 2,395 20.4 1.00 1.77*** 

Tested for HIV3

12,651 

(11.8, 13.3) 

42.3 1,742 

(17.4, 23.7) 

40.5 1.00 

(1.43, 2.19) 

0.88 

3 HIV risk behaviors 12,908 

(41.2, 43.4) 

4.1 1,731 

(36.9, 44.2) 

6.9 1.00 

(0.74, 1.04) 

1.74** 

(3.6, 4.6) (5.1, 9.4) (1.18, 2.56) 

RDD = random-digit dial; DSF = Delivery Sequence File; AOR = adjusted odds ratios; CI = confidence interval 

Significance: * p<.05, ** p<.01, *** p < .001 

Note: includes data from CA, NC, NJ, TX and WA. Data are weighted to account for sample design and post-stratified to sex-age 

totals for each state. The final weights were ratio adjusted to equalize the number of cases across states. 

1 Models are adjusted for respondents’ state of residence, sex, race, age, education, and having health care coverage. 

2 Reference category. 

3

 Questions not asked of respondents age 65 years or older 
Table 3. Adjusted odds ratios for effects of mail survey features on self-reports of health conditions and risk factors 

Surname on mailing label Follow-up postcard Second questionnaire 

mailing

Within household 

respondent selection 

1 

Health condition / 

risk factor n 

No

name

match 

AOR1 

Name

matched –

not used 

AOR 

(95% CI) 

Name

matchedused

AOR 

(95% CI) 

No

postcard 

AOR1 

Postcard 

sent 

AOR 

(95% CI) 

No

second 

mailing 

AOR1 

Second 

mailing 

sent 

AOR 

(95% CI) 

Any

adult 

AOR1 

Next 

Birthday

AOR 

(95% CI) 

All Adults 

AOR 

(95% CI) 

Asthma 2.941 1.00 0.82 

0.61, 1.08) 

0.71* 

0.53, 0.95) 

1.00 1.08 

(0.87, 1.33) 

1.00 0.93 

(0.75, 1.16) 

1.00 0.92 

(0.70, 1.22) 

0.82 

( ( (0.63, 1.06) 

Diabetes 2,984 1.00 1.13 

(0.79, 1.64) 

1.16 

(0.80, 1.68) 

1.00 0.93 

(0.73, 1.18) 

1.00 1.14 

.88, 1.47) 

1.00 0.94 

(0.68, 1.30) 

0.96 

(0 (0.72, 1.28) 

High blood pressure 2,958 1.00 0.98 

(0.70, 1.12) 

0.73* 

0.57, 0.93) 

1.00 1.16 

(0.98, 1.37) 

1.00 1.15 

(0.97, 1.36) 

1.00 1.03 

(0.83, 1.28) 

0.91 

( (0.75, 1.11) 

Obese (BMI > 30) 2,910 1.00 0.80 

(0.62, 1.02) 

0.82 

(0.64, 1.06) 

1.00 1.11 

(0.93, 1.33) 

1.00 1.10 

(0.92, 1.32) 

1.00 0.97 

(0.77, 1.22) 

0.89 

(0.72, 1.10) 

Current smoker 2,956 1.00 0.83 

(0.63, 1.11) 

0.77 

(0.58, 1.03) 

1.00 1.25* 

(1.01, 1.54) 

1.00 1.23 

(0.99, 1.54) 

1.00 0.99 

(0.76, 1.30) 

0.88 

(0.70, 1.13) 

Binge drinking 2,951 1.00 0.81 

(0.60, 1.09) 

0.97 

(0.72, 1.31) 

1.00 0.93 

(0.75, 1.15) 

1.00 0.85 

(0.68, 1.06) 

1.00 1.01 

(0.76, 1.34) 

0.97 

(0.75, 1.26) 

Tested for HIV2

2,143 1.00 0.75* 

(0.58, 0.96) 

0.73* 

(0.56, 0.95) 

1.00 1.15 

(0.95, 1.39) 

1.00 1.07 

(0.88, 1.29) 

1.00 0.99 

(0.78, 1.27) 

0.77 

(0.65, 1.01) 

HIV risk behaviors2 2,130 1.00 0.38*** 

(0.22, 0.64)

0.33*** 

(0.19, 0.57) 

1.00 1.16 

(0.75, 1.78) 

1.00 0.56** 

(0.36, 0.86) 

1.00 1.34 

(0.78, 2.30) 

0.96 

(0.55, 1.65) 

AOR = adjusted odds ratios; CI = confidence interval 

Significance: * p<.05, ** p<.01, *** p < .001 

Note: includes data from CA, IL, NC, NJ, TX and WA. Data are weighted to account for sample design and post-stratified to sex-age totals for each state. The 

final weights were ratio adjusted to equalize the number of cases across states. Logistic regression models are adjusted for respondents’ state of residence, sex, 

race, age, education, and having health care coverage. 

Reference category. 

2

 Questions not asked of respondents age 65 years or older 
Health condition / 

risk factor n 

Landline 

only 

AOR1

Landline and 

cell phone 

AOR 

(95% CI) 

Cell phone 

only 

AOR 

(95% CI) 

Asthma 2,795 1.00 0.84 1.08 

(0.63, 1.11) (0.63, 1.85) 

Diabetes 2,815 1.00 0.86 1.59 

(0.64, 1.16) (0.86, 2.93) 

High blood pressure 2,795 1.00 0.95 0.99 

(0.77, 1.19) (0.63, 1.57) 

Obese (BMI > 30) 2,758 1.00 0.95 1.05 

(0.74, 1.21) (0.65, 1.69) 

Current smoker 2,813 1.00 0.70* 1.06 

(0.53, 0.93) (0.66, 1.71)

Binge drinking 2,805 1.00 1.56* 1.90* 

Tested for HIV2

2,054 1.00 

(1.10, 2.21) 

1.02 

(1.10, 3.28)

1.35 

2 HIV risk behaviors 2,040 1.00 

(0.75, 1.38)

1.22 

(0.82, 2.21) 

3.95** 

(0.59, 2.55) (1.64, 9.50) 

Table 4. Adjusted odds ratios for effects of type of household 

telephone(s) on self-reports of health conditions and risk factors 

Type of household telephone(s)

AOR = adjusted odds ratios; CI = confidence interval 

Significance: * p<.05, ** p<.01, *** p < .001

Note: includes data from CA, IL, NC, NJ, TX and WA. Data are weighted to account 

for sample design and post-stratified to sex-age totals for each state. The final weights 

were ratio adjusted to equalize the number of cases across states. Logistic regression 

models are adjusted for respondents’ state of residence, sex, race, age, education, and 

having health care coverage. 

1 Reference category. 

2

 Questions not asked of respondents age 65 years or older