PSEUDO -‐ DYADIC INTERACTION 3 Pseudo-dyadic “ interaction ” on Amazon ’ s Mechanical Turk Amazon ’ s Mechanical Turk

Psychological researchers have begun to utilize Amazon’s Mechanical Turk (MTurk) marketplace as a participant pool. Although past work has established that MTurk is well-suited to examining individual behavior, pseudo-dyadic interactions, in which participants falsely believe they are interacting with a partner, are a key element of social and cognitive psychology. The ability to conduct such interdependent research on MTurk would increase the utility of this online population to a broad range of psychologists. The current research therefore attempts to qualitatively replicate well-established pseudo-dyadic tasks on MTurk in order to establish the utility of this platform as a tool for researchers. We find that participants do behave as if a partner is real, even when doing so incurs a financial cost, and that they are sensitive to subtle information about the partner in a minimal-groups paradigm, supporting the use of MTurk for pseudo-dyadic research. Keywords: interaction, internet, cooperation, social influence, interdependent decision making PSEUDO-­‐DYADIC INTERACTION 3 Pseudo-dyadic “interaction” on Amazon’s Mechanical Turk Amazon’s Mechanical Turk (MTurk) website, in which “workers” complete brief online tasks for small payments, has become an enormously popular pool of participants (e.g., Mason & Suri, 2012; Rand, 2011), following papers validating the population and the quality of data it provides for survey research (Buhrmester, Kwang, & Gosling, 2011; Paolacci, Chandler & Ipeirotis, 2010). MTurk addresses several key needs of researchers: a large, relatively diverse sample (compared to most undergraduate participant pools) that can provide data quickly and at low cost. Although MTurk is well-suited for survey and experimental research focused on individuals, as noted above, expanding the range of studies that can appropriately be run on MTurk would be quite useful to the field. In particular, many psychologists wish to investigate behavior in an interactive or social context, such as judgment and decision-making (JDM) researchers interested in interdependent decision-making or social psychologists interested in interpersonal interactions. Such studies often require participants to believe that another person is currently interacting with them (e.g., Pillutla & Murninghan, 1996; Sanfey, Rilling, Aronson, Nystrom, & Cohen, 2003; Williams, Cheung, & Choi, 2000). This research therefore aims to examine whether research involving ostensibly dyadic tasks can be conducted on MTurk with confidence. We wanted to determine whether participants behave as if their “partners” are real by establishing that the types of effects previously found in lab research will also occur in online studies with MTurk participants. We focus specifically in the current studies on pseudo-dyadic tasks, research that only purports to involve interaction, rather than attempting to actually allow direct interaction between participants. Research involving only a simulacrum of interaction is quite common throughout psychology. In many studies, although participants may be in the lab simultaneously PSEUDO-­‐DYADIC INTERACTION 4 with another person (either a confederate or another participant), the content of the ostensible interaction is in fact predetermined by the experimental protocol. Paradigms that involve ostensible interaction are widely used in the social rejection literature (see Blackhart et al., 2009 for a meta-analysis). One commonly used paradigm of social rejection, Cyberball (Williams, Cheung, & Choi, 2000), involves participants playing a computerized game of “catch” that is ostensibly with other participants but in fact is entirely pre-programmed. Similarly, research on economic decision-making frequently asks participants to make and respond to offers from phantom partners. For example, participants may be asked to respond to bargaining offers that are ostensibly made by a partner, but are instead predetermined by the experimenter (e.g., Pillutla & Murninghan, 1996; Sanfey et al., 2003). Alternatively, participants may interact over multiple iterations or rounds of an economic game with an ostensible partner, when in fact the partner’s behavior is a computerized response determined by the decisions of the participant (e.g., a “tit-for-tat” strategy; van Lange, Ouwerkerk & Tazelaar, 2002). Using pseudo-dyadic tasks allows for non-interdependence in participant responses, enabling the testing of specific hypotheses with fewer participants. Additionally, conducting truly interactive studies over MTurk is quite complex and requires advanced programming and networking skills (Siddharth & Winter, 2012). Given both the frequency of pseudo-interaction in the study of social behavior and the technological challenges involved in actually linking participants, our investigation focused on pseudo-interaction tasks in which there was never a real “partner” with whom a participant would interact. Instead, we used an online survey system (Qualtrics.com) that allows the presentation of screens to be timed, allowing us to lead participants to believe that they were waiting to be connected to and communicate with another person, when in reality the inputs from PSEUDO-­‐DYADIC INTERACTION 5 the “partner” were pre-determined by the experimenters and the participants were in no way linked to another person. To ensure that participants would be debriefed about the true nature of the study (i.e., that there was no partner), we linked payment on the MTurk system to a “completion code” that participants received on the debriefing screen; so that participation was fully voluntary, participants were told that they could reach this screen without answering any questions. To determine whether such pseudo-interactive behavior on MTurk is believable to participants and represents a reasonable approximation of that which researchers could expect in the lab, we conducted replications of well-established dyadic tasks. By examining tasks in which a clear and consistent effect has emerged in past research, we felt that null effects would be meaningful failures of MTurk as a venue for these tasks. Conversely, we believed that establishing reliable effects that descriptively parallel those in the lab would support the appropriateness of MTurk for these types of studies. We were therefore interested in establishing the qualitative assumption of interactivity by participants, rather than in conducting a full quantitative comparison to establish identical effect sizes and sampling distributions. In Study 1, we examined informational social influence by a supposed partner, which we compared to an anchoring effect from an estimate explicitly provided by the computer. In Studies 2-4, we raised the stakes for participants by examining behavior in economic tasks in which cooperation with a partner would lead the participant to forgo some amount of bonus payment. Finally, in Study 4, we also examined whether an established moderator of social behavior, ingroup versus outgroup status in a minimal groups paradigm, would likewise be able to influence social behavior on MTurk. Following the recommendations of Simmons, Nelson, & Simonsohn (2011) for minimizing the likelihood of false positive results, the studies below report all collected variables PSEUDO-­‐DYADIC INTERACTION 6 and all experimental conditions; moreover, the reported studies represent all but one of the pseudo-dyadic studies we have attempted to date on MTurk. (We previously conducted a version of Study 2 with only one of the conditions, for which the effects were comparable.) Sample sizes were set a priori as noted in each study; the obtained N is slightly greater in Studies 1, 2, and 3 because some participants did not enter their payment code on MTurk, leading to additional “workers” being recruited by the automated system. Study 1 Informational social influence, in which individuals alter their behavior because of the perceived value of the information provided by others, is one of the most classic social psychological phenomena (Deutsch & Gerard, 1955). Initially demonstrated in Sherif’s (1935) study using the autokinetic effect, it has been shown to influence a wide range of decisions, from eyewitness recall in suspect lineups (Steblay, 1997) to product purchasing (Bearden & Etzel, 1982). Study 1 aimed to demonstrate this effect in participants on MTurk by comparing the effect of a low (versus high) anchor provided by a “partner” or by the computer on numeric estimates generated by participants. While even randomly generated anchors have an effect on numeric estimates (Tversky & Kahneman, 1974), the effect should be stronger if the anchor is believed to come from another person. Method 102 participants (a priori N = 100) were recruited from Amazon’s Mechanical Turk (MTurk) for a study with the description “Decision-making Study: This study examines how people make and react to decisions” for which they were paid 25 cents. Participation was limited to participants located in the United States with a past approval rating in the MTurk system above 95%. PSEUDO-­‐DYADIC INTERACTION 7 Participants were told that they would be asked to complete an estimation task, and were randomly assigned to only the “computer anchor” or “partner estimate” condition. Participants in the “computer anchor” condition were informed that the computer would randomly generate a starting value for the estimation task. When participants advanced the screen, the message “Please wait while the computer generates a random number. ” appeared on screen for 31 seconds. Participants in the “partner estimate” condition were told that they would be paired with another participant, and that they would provide estimates sequentially, with the participant who went second being informed of the partner’s estimate. When participants advanced the screen, the message “Please wait while the computer connects to another particip