Getting Physicians to Respond: The Impact of Incentive Type and Timing on Physician Survey Response Rates

Survey research is a critical tool in health services research necessary to assess clinical practice patterns that influence the implementation of evidence-based care in an era of comparative effectiveness research. However, survey research, especially among physicians, suffers from lackluster participation relative to their nonphysician counterparts, with response rates to surveys of the former about 10 percentage points lower than surveys of the latter, on average (Asch, Jedrziewski, and Christakis 1997). Although emerging evidence indicates that response rates are poorly correlated with response bias (Groves 2006; Groves and Peytcheva 2008;), in our experience few general medical journals are willing to publish physician surveys with response rates below 50 percent. A recent survey of scientific journal editors showed that approximately 90 percent believe response rate is at least somewhat important in publication decision making (Carley-Baxter et al. 2009). In addition, virtually none of the surveyed editors indicated they have changed their response rate standards in the past 10 years, despite widespread evidence of decreasing response rates to both general population surveys (Steeh et al. 2001; de Leeuw and de Heer 2002; Curtin, Presser, and Singer 2005; Berk, Schur, and Feldman 2007;) and surveys of physicians (Cummings, Savitz, and Konrad 2001; Cull et al. 2005;) over that timeframe. Commonly, survey researchers face the difficult decision of whether to send an additional wave of surveys in hopes of surpassing a key response rate threshold for the general medical literature such as 50 percent or 60 percent. Knowing how to do so judiciously remains an important challenge. Optimizing physician response rates without adversely influencing response bias also remains a key concern among health services researchers. Physician participation in surveys has been shown to be effectively increased through the use of incentives, especially when the incentive is monetary and offered in advance of completing a survey (prepaid) versus being offered contingent on completion of a survey (promised) (Asch, Christakis, and Ubel 1998; Kellerman and Herold 2001; VanGeest, Johnson, and Welch 2007; Flanigan, McFarlane, and Cook 2008;). However, the evidence supporting prepayment over promised monetary incentives is undermined by the lack of controlled experiments testing the two approaches among physicians. As such, the literature can only be construed as suggestive on this matter. Moreover, the extant literature offers little guidance on whether the effectiveness of prepaid monetary incentives varies by the method of its delivery (i.e., cash versus check). Finally, from a research subject protection perspective, immediate incentives (check or cash) that do not require the participant to disclose a tax identification number such as a social security number (SSN) allow participants to better protect their personal privacy and maintain confidentiality—the primary risks associated with survey research. Nevertheless, many institutions require tax identification numbers as a condition of dispensing funds to research participants. This can severely limit the mode and timing of remuneration—each of which have significant implications for response rates. In an era of identity theft, it is important to determine the best ways to both optimize response rates by offering remuneration to research participants who receive small, one-time survey incentives while protecting their privacy. To better understand the effects of payment timing (prepaid versus promised), form of payment (cash versus check), and having to provide an SSN on response rate, response bias, and item nonresponse, we embedded a randomized incentive experiment crossing several of these factors within the third-wave mailing to a nationally representative sample of U.S. physicians. We hypothesized that immediate incentives offered without the provision of an SSN would increase response rates beyond the rates from promised incentives, that cash is more effective than check as an immediate incentive, and that promised incentives not requiring provision of an SSN would yield higher response rates than those requiring it.