Data collection in consumer research has progressively moved away from traditional samples (e.g., university undergraduates) and toward Internet samples. In the last complete volume of the Journal of Consumer Research (June 2015-April 2016), 43% of behavioral studies were conducted on the crowdsourcing website Amazon Mechanical Turk (MTurk). The option to crowdsource empirical investigations has great efficiency benefits for both individual researchers and the field, but it also poses new challenges and questions for how research should be designed, conducted, analyzed, and evaluated. We assess the evidence on the reliability of crowdsourced populations and the conditions under which crowdsourcing is a valid strategy for data collection. Based on this evidence, we propose specific guidelines for researchers to conduct high-quality research via crowdsourcing. We hope this tutorial will strengthen the community's scrutiny on data collection practices and move the field toward better and more valid crowdsourcing of consumer research. Keywords: Mechanical Turk, MTurk, crowdsourcing, data collection, sampling A cademic consumer research strongly depends on the availability of study participants. Theories of consumer behavior typically require tests with human participants, and researchers often choose their samples based on convenience. For decades, this translated into an overwhelming reliance on undergraduate students, and the Journal of Consumer Research (JCR) hosted several debates on the external validity of these investigations (Calder, Phillips, and Tybout 1981 , 1982 , 1983 Ferber 1977; Lynch 1982 Lynch , 1983 McGrath and Brinberg 1983; Peterson 2001; Wells 1993) . In recent years, however, consumer researchers have increasingly turned to the Internet to recruit study participants and collect data, and in particular to crowdsourcing. On websites such as Mechanical Turk (MTurk) and Prolific, researchers act as "employers" and hire and compensate "workers" to participate in computerized tasks (e.g., surveys, choice tasks, and/ or simulated shopping environments). Crowdsourcing brings a new meaning to convenience sampling. These platforms grant unprecedented efficiencies, providing researchers with participants who can be accessed at any point in time, are more demographically diverse, and are less expensive to reach than traditional research participants. Consistent with trends in other social sciences, consumer research is now routinely, if not by default, conducted using online marketplaces. Currently, the most prevalent crowdsourcing destination is MTurk, a marketplace launched in 2005 by Amazon. Over 15,000 published papers referenced MTurk in the past 10 years (Chandler and Shapiro 2016 196 well be the most represented participant pool in the history of consumer research. With its convenience, crowdsourcing has also brought its share of skepticism and controversy. Echoing early concerns with online research (Kraut et al. 2004) , some researchers worry about the impossibility of scrutinizing participants' behavior-participants might be multitasking or be interrupted during the study. Further, crowdsourced participants self-select into studies and can quit at any time, and samples may vary between studies and between conditions within a study as a result of arbitrary choices in the design and sampling process (Casey et al. 2017; Zhou and Fischbach 2016). Thus, there is concern that MTurk workers may not provide reliable data or be particularly representative of real-world consumers-a similar criticism levied for decades on the use of college students in consumer research (Calder et al. 1981 (Calder et al. , 1982 (Calder et al. , 1983 Ferber 1977; Lynch 1982 Lynch , 1983 McGrath and Brinberg 1983; Peterson 2001; Petty and Cacioppo 1996; Wells 1993) . Perhaps more concerning to critics is the validity of data obtained from participants who have accumulated experience with social science studies (Pham 2013). At a more philosophical level, there is concern that the efficiency of online samples might increase the attractiveness of research programs and paradigms that can be easily conducted with the crowdsourcing method (e.g., scenarios), at potential detriment to research that is important but difficult to crowdsource (Pham 2013) . These concerns are legitimate. While the option to crowdsource empirical investigations has great efficiency benefits for both individual researchers and the field, it also poses new challenges for how research should be designed, conducted, and analyzed by researchers, and evaluated by editors, reviewers, and readers. In this tutorial, we assess the evidence on the reliability and validity of crowdsourced populations and the conditions under which crowdsourcing is a valid strategy for data collection, with the goal to establish valid crowdsourcing as a data collection strategy in consumer research. First, after a brief overview of MTurk, we discuss how crowdsourcing provides advantages to individual researchers and the consumer research field. We argue that MTurk and similar websites can improve not only the convenience, but also the quality, of consumer research-if used appropriately. Second, drawing from methodological research on crowdsourcing, we address the substance and magnitude of concerns associated with the use of crowdsourced samples. Third, we offer specific guidelines for consumer researchers to maximize the advantages of crowdsourcing while attenuating the methodological concerns raised in the literature. Though we will predominantly focus on MTurk-the most popular crowdsourcing destination among behavioral researchers-our findings by and large apply to any crowdsourcing solution that is or will become available in the foreseeable future. As opportunities will flourish to harness the advantages of online samples, we hope our tutorial will trigger new contributions on the methodology of data collection in consumer research.
Research on the demographics of workers on MTurk is closely linked with questions concerning the representativeness of MTurk samples and their suitability for different research purposes (e.g. REF ).
Crowdsourcing Consumer Research
Journal of Consumer Research
Business