Conducting an Internet-Based Survey: Benefits, Pitfalls, and Lessons Learned

During the last 10 years, increasing numbers of researchers have been shifting to the use of Internet-based survey methodologies from traditional recruitment strategies and paper-and-pencil data collection methods. One reason is that the Internet has become accessible to "ordinary people," not just those who are computer savvy (Wellman, 2004). In 2007, 61.7% of all U.S. households had Internet access inside the home, and 71.0% had access outside the home (U.S. Department of Commerce, 2009). Furthermore, user-friendly survey software has streamlined the questionnaire design process and simplified the process of collecting and inputting data. Just as questions about sampling, recruitment, data collection, and response rates emerged when researchers began to use mail and telephone surveys, researchers have been trying to determine the best way to conduct surveys using Internet technology (Schonlau, Fricker, & Elliot, 2002). Although Internet-based methods have some important advantages, researchers who are used to traditional survey methods may encounter new challenges when using them. In this article, we build on the extant literature on Internet-based survey methods to examine some advantages and challenges associated with their use in social work research. We discuss the problems we encountered, mistakes we made, and solutions we found when using Internet methods for the first time to recruit participants and collect data, paying particular attention to sample recruitment, the use of incentives, and ethical issues. Our goal is to help social work researchers understand how planning and conducting a survey using Internet methods may differ from doing so with traditional methods and how taking certain steps could help to avoid problems that we encountered. The study on which this article is based examined stressful life events, coping, and risky sexual behavior in a nationwide convenience sample of 297 self-identified gay and bisexual men in the United States. We used SurveyMonkey (http://www. surveymonkey.com/), an online survey software program, to design and host our survey. There are a variety of similar programs, including Zoomerang (http://www.zoomerang.com/) and Zarca Interactive (http://www.zarca.com/). These programs offer researchers a variety of questionnaire design formats that are fairly easy to use, and they make the coding and cleaning of data nearly obsolete. Participants' responses to each questionnaire item are recorded electronically, and summary data are immediately available for review by the researcher. Collected data may be downloaded in a variety of formats, including Excel spreadsheet format, which then can be easily converted into SPSS. SAMPLING AND RECRUITMENT Our sampling strategy involved posting requests for participation on the Web sites of three gay, lesbian, bisexual, and transgender (GLBT) community agencies, in Craigslist's classified ads and discussion forums, and on two commercial Web sites that focus on health and lifestyle issues for individuals living with HIV/AIDS. Use of a convenience sample was appropriate for our study for two reasons. First, a limited budget prevented us from using more costly probability-based methods. Second, convenience samples are frequently used to recruit GLBT individuals and other members of hard-to-reach populations because reliable sampling frames are generally not available (Koch & Emrey, 2001). GLBT populations may be hard to reach because of social isolation or perceived or actual discrimination (Sullivan & Losberg, 2003). However, convenience samples such as ours are likely to contain selection biases. Thus, it is important to report information about the venues where the study advertisements were posted so as to provide general information about some of the likely characteristics of respondents obtained from these venues (Eysenbach, 2004). As with all nonprobability samples, there is limited ability to generalize findings to the populations from which they were drawn (Eysenbach, 2004; Kraut et al. …