Tests of methods to improve response to physician surveys 

Catharine W. Burt, Ed.D., and David Woodwell 

Centers for Disease Control and Prevention 

National Center for Health Statistics 

Division of Health Care Statistics

3311 Toledo Road, Suite 3409 

Hyattsville, MD 20782 

cburt@cdc.gov

dwoodwell@cdc.gov

Introduction

Health surveys targeting physicians historically have had difficulties in obtaining high response rates. Response rates for 

physician surveys are routinely in the 40-50% range.123 Response rates to the National Ambulatory Medical Care Survey 

(NAMCS), a leading survey of visits to office-based physicians, while larger than most physician surveys have nevertheless

been declining since 1985 (from ~75-65%). Research had shown that some improvements in response rates with physicians

were found when monetary incentives were employed.45 Cooperation theory would suggest that if the sampled physicians

understood the importance of the survey, they would be more likely to cooperate because there would be a benefit to 

themselves as well as the survey.6 Despite the difficulty in achieving high response rates, researchers and policy makers are 

always requesting more questions on these surveys which increase the response burden for individual physicians. NCHS

conducted several studies between 2000 and 2002 to try to gauge the impact of different methods on physician response to

the NAMCS. One of the goals of the survey is to keep response above 70% at a minimum. The lower the response rate, the 

more likely that survey estimates may be biased in the direction of responding physicians. New Office of Management and

Budget (OMB) policy indicates that bias studies must be performed when the response rate falls below 80%.7 While it is

doubtful that physician response could be raised as high as 80%, it was hoped that efforts could increase NAMCS response to

at least 70% so that nonresponse bias would be minimized. 

Background of the NAMCS

The NAMCS is conducted annually by CDC’s National Center for Health Statistics in order to produce estimates of

physician office utilization including patient characteristics such as age, gender, and race; visit characteristics such as 

expected source of payment, patient complaints or reasons for visit, diagnoses rendered; and diagnostic services and 

treatments ordered or provided, including medications prescribed. Additionally, the NAMCS provides national estimates of

practice characteristics such as size of practice, physician specialty, ownership, etc. The Census Bureau is the data collection 

agent for the survey and the collected data are centrally processed including medical coding by Constella Group, Inc. In the 

NAMCS, a multi-stage probability sample of non-federal office-based physicians, each sampled physician is mailed an 

introductory letter from the Director of NCHS explaining the purpose of the survey, the authority with which NCHS collects

the data (Public Health Service Act), and that participation is voluntary. The letter also states that a Census Bureau Field 

Representative (FR) will be calling to make an appointment. At the face-to-face interview, the FR asks the physician some

eligibility questions and other questions about the practice. The physician is then asked to complete a one-page form for each

of about 30 sampled patient visits to the office during the randomly-assigned reporting week. The Patient Record Form (PRF)

collects non-identifying demographic information about the patient and the content of the visit. Copies of the data collection

materials can be viewed on the participant WEBsite: www.cdc.gov/NAMCS . The induction interview with the sampled

physician (or staff) takes about 15 minutes to complete, while each PRF takes about 2 minutes to complete. About one-third 

of the time, the physician requests that the FR abstract the data for the PRF from the medical record. The physician receives

no payment as incentive to participate or fee to perform the record abstraction. During the induction interview, the FR 

emphasizes the importance of the survey and why the physician’s response is critical. The FR provides materials to the 

physician (or office staff) about the survey including a list of articles published in medical and public health journals that use

NAMCS data, a fact sheet of key estimates for physicians overall and for the sampled physician’s specialty, and the annual 

NAMCS summary report. FRs will make up to 6 call attempts and senior FRs are used to attempt refusal conversions. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Methods 

Physicians in the normal NAMCS production sample were randomly assigned to treatment and control groups in three 

separate studies to test the effects of the following on NAMCS response rates: 1) inclusion of a motivational insert with the 

introductory letter containing Q & A’s about the survey and its importance, 2) offering monetary ($50) or token gift ($15

value of candy, flowers, or other token) incentive at the time of survey induction and 3) length of data collection instrument ( 

1 or 2 sided legal sized page). The short form consisted of 70 items on a single legal page and the long form consisted of 140

items on a 2-sided legal page. For each study an attempt was made to ensure that sampled physicians who happen to be in the

same medical practice were assigned to the same study group. Weighted response rates were compared between the 

treatment and control groups for each study separately. Response rate calculation was based on the number of responding 

physicians divided by the number of in-scope physicians (responding and refusing physicians). Table 1 provides the sample

sizes and the dates of the three studies. 

Table 1: Studies of methods to improve NAMCS response rates 

Study Sample size Study Dates 

Motivational insert 513 Insert 

439 Control

July-December, 2000 

Incentive 456 Money

401 Token gift

418 Control

April-December, 2002 

Form length 941 Short form

969 Long form

January-December, 2001

Results 

Motivational insert 

The response rate for the motivational insert group was not significantly different from the control group (68.2% se=2.5 vs. 

64.3% se=2.5). Furthermore, there were no differences in response rates by physician characteristics (e.g., geographic

region, metropolitan statistical area (MSA) status, specialty, type of practice, physician age- data not shown). 

Incentive test

Response rates did not vary across the 3 incentive conditions (monetary: 72.7 % se= 2.5, token gift: 67.6% se= 2.8, and

control: 72.7% se=2.6, respectively). Only 2 physician characteristics were associated with differential response rates in the

incentive groups: physicians in the West had lower response rates for the 2 incentive groups (56.6% for token and 67.3% for 

monetary) compared to the control group (79.8%) and response by physicians in surgical specialties was influenced more by

a monetary gift (77.2%) compared to a token gift (59.8%). 

Form length 

The response rate for the short form group was significantly higher than that of the long form group (67.6% se=2.0 vs. 61.9% 

se=2.1). These differences were driven primarily by differential response rates observed in the northeastern and western

regions, for physicians in solo practice, for those working in MSAs, and for physicians in general and family practice (data

not shown). More detailed results for this study may be found elsewhere.8

Discussion 

Information from these three studies indicated that we could not positively affect physician response but we could negatively 

affect it. The first 2 studies indicated that use of motivational inserts in the introductory letter and use of incentives were not 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

helpful in improving response. The third study indicated that increasing the burden could reduce response rates. There were

some differential results where the monetary incentive did increase response rates for surgeons, but it is hard to implement a 

policy where surgeons would be offered the incentive and other physicians would not. Similarly, with regard to increased

burden, some physicians were not negatively affected (i.e., physicians in the south and mid-western regions, and physician in

group practices). But operationally speaking, we must implement methods that will produce the best national results. 

These studies were all conducted prior to the implementation of the Privacy Rule in the Health Insurance Portability and 

Accountability Act of 1996 (HIPPA) which went into effect April 14, 2003. The motivational insert was revised to include

information about the Privacy Rule, approval of the survey by CDC’s institutional review board, data use agreements, and 

accountability requirements. We decided that even though the insert did not increase response, it may calm physicians’ fear

about whether they could comfortably participate in the NAMCS in light of the new privacy requirements. 

While there was no difference in response rates among the 3 incentive groups, it is important to note that during the year that

the incentive test was conducted, each of the 3 groups had a response rate higher than the overall response rate in previous

years. It has been hypothesized that the use of incentives for even some of the cases made the FRs feel more comfortable in 

seeking cooperation, or eased their conversion efforts for the incentive groups so they could concentrate on the control group.

Feedback from the FRs indicated that the logistics involved in actually giving the money to the physicians was cumbersome

and that several of the doctors that participated in the survey refused to accept the incentive. The FRs preferred the token gift 

which could be flowers, candy, fruit, donuts, etc, because that was a gift that the people actually doing the work (i.e., the 

office staff) could appreciate. 

NCHS works continuously to find ways to increase survey response among physicians. At present we are preparing a CME 

course about the survey and how it is used in medical and public health research that any doctor could complete, but unless

you are a NAMCS sampled physician you are unlikely to know about it. We are also working with the Association of

American Medical Colleges (AAMC) to implement an educational unit on the role of surveys in providing important data for 

health services research. 

In light of our inability to consistently raise response rates to 70%, we have initiated other avenues to decrease nonresponse 

bias. The NAMCS has always considered the physician specialty, geographic region, and metropolitan statistical area (MSA) 

status in the nonresponse adjustment for the weighting process. Beginning in 2001, the NAMCS induction interview 

requested minimal information about the practice for physicians who would refuse to provide the PRF data such as number of

weekly visits and number of office locations. Starting in 2003, we integrated the visit volume and number of weeks a doctor 

sees patients into the nonresponse adjustment factor as physicians with larger weekly volumes were increasingly more likely 

to refuse. 

The first two studies reported here were limited in their sample size which, because of the clustered nature of the sample 

design, meant that the power to detect a 5% change in response rates was not as high as desired. The test of form length just

barely had sufficient power (95%) to detect a 5% change in response rates. The sample sizes were almost double that used in 

the first 2 studies. 

Conclusions 

st

Physicians will continue to be a very difficult group to survey in the 21 century. The policy implications include that 

decisions based on health care delivery estimates from surveys of physicians in the US may continue to be compromised

even further. The focus may need to switch from improving response rates to estimating and controlling for nonresponse

bias. Use of monetary incentives may need to be differentially applied to physicians; which raises ethical dilemmas. 

Key Terms: NAMCS, office-based physicians, response rate, and methodological studies 
 

 

 

 

 

 

 

 

 

 

 

 

 

References 

1

Martin B. Don't survey physicians! Chicago, IL: Center for Health Services Research and Development, American Medical 

Association; 1974. 

2

Shosteck H, Fairweather WR. Physician response rates to mail and personal interview surveys. Public Opinion Q 1979; 43:

206-217. 

3

 Sudman S. Mailed survey of reluctant professionals. Eval Rev 1985;9:349-60 

4

Berk ML, Edwards WS, Gay NL. The use of a prepaid incentive to convert nonresponders on a survey of physicians. 

Evaluation & the Health Profession 1993; June: 239-245 

5

Halpern SD, Ubel PA, Berlin JA, Asch DA. Randomized Trial of $5 Versus $10 monetary incentives, envelope size, and

candy to increase physician response rates to mailed questionnaires. Medial Care 40; 9: 834-39. 

6

Tjosvold, D. Cooperation theory and organizations. Human Relations. 1984; 37: 743-767

7

Harris-Kojetin OMB Guidelines for Surveys. 2004 

8

Hing E, Schappert S, Burt C, Shimizu I. Effects on form length and item format on response patterns and visit estimates: 

NAMCS and NHAMCS, 2001. National Center for Health Statistics. Vital Health Stat 2(139). 2005