not be self-evident. Although perfect simulation is not necessary to retain internal validity while increasing external validity, how best to simulate selection behavior is not at all self-evident. For the most part, our proposed design avoids one set of potential problems. The condition does not ask subjects to predict future behavior, recall past behavior, or describe hypothetical, counterfactual behavior. Nor does it ask them to justify or explain their choices. Our proposed design avoids errors that might arise from these reflective and simply invites subjects to make a choice that should resemble its equivalent. One question that any user of a self-selection experiment will want to consider is how blind the subject should be to selection into the treatment. As a general matter, economists and psychologists could not disagree more, with the former answering “not blind at all” and the latter answering “completely blind.” Staying with the ads example, to simulate self-selection, an experimenter might simply give subjects the option of watching a negative campaign ad, a positive campaign ad, or an ad that has no political content. Giving subjects that choice would, however, break with past precedent of disguising the experiment's focus. Insofar as one worries that subjects react to ads differently if they know that the experiment is focused on ads, an option is to alter the self-selection item. Because candidates frequently air their campaign advertisements during the evening news hour, researchers might invite subjects to choose whether or not to watch a newscast. That implementation might mimic accidental self-selection, but it would not account for those who systematically miss real-world advertisements because, when these air, they leave the room, change the channel, or shut off their brains. To that end, the study might also give subjects the ability to fast through video material, change channels, turn the TV off, or walk about. Generally, there will be tradeoffs between alternative
Dunning, Thad. 2008. “Improving Causal Inference: Strengths and Limitations of Natural Experiments.” Political Research Quarterly 61 (2): 282-93. Finkel, Steven E. and John G. Geer. 1998. “A Spot-Check: Casting Doubt on the Demobilizing Effect of Attack Advertising.” American Journal of Political Science 42 (2): 573-95. Fisher, Ronald Aylmer. 1935. The Design of Experiments. Edinburgh, UK: Oliver and Boyd. Gaines, Brian J., James H. Kuklinski, and Paul J. Quirk. 2007. “The Logic of the Survey Experiment Reexamined.” 2007. Political Analysis 15 (1): 1-20. Garfinkel, Alan. 1981. Forms of Explanation: Rethinking the Questions in Social Theory. New Haven, CT: Yale University Press. Gerber, Alan S. and Donald P. Green. 1999. “Does Canvassing Increase Voter Turnout? A Field Experiment.” Proceedings of the National Academy of Sciences of the United States of America 96 (19): 10939-42. Gerber, Alan S. and Donald P. Green. 2008. “Field Experiments and Natural Experiments.” In Oxford Handbook of Political Methodology. Janet M. Box-Steffensmeier, Henry E. Brady, and David Collier, eds. Oxford, UK: Oxford University Press, 357-81. Geer, John G. 2006. In Defense of Negativity: Attack Ads in Presidential Campaigns. Chicago, IL: University of Chicago Press. Goldstein, Ken and Paul Freedman. 2002. “Campaign Advertising and Voter Turnout: New Evidence for a Stimulation Effect.” Journal of Politics 64 (3): 721-40. Granato, James, Melody Lo, and M. C. Sunny Wong. 2010. “A Framework for Unifying Formal and Empirical Analysis.” American Journal of Political Science 54 (3): 783-97. Heckman, James J. and Jeffrey A. Smith. 1995. “Assessing the Case for Social Experiments.” Journal of Economic Perspectives 9 (2): 85-110.
Holland, Paul W. 1986. “Statistics and Causal Inference.” Journal of the American Statistical Association 81 (396): 945-60. Hovland, Carl I. 1959. “Reconciling Conflicting Results Derived from Experimental and Survey Studies of Attitude Change.” American Psychologist 14 (1): 8-17. Iyengar, Shanto and Donald R. Kinder. 1987. News that Matters. Chicago, IL: University of Chicago Press. Jackman, Simon and Lynn Vavreck. 2009. “The Magic of the Battleground: Uncertainty, Learning, and Changing Environments in the 2008 Presidential Campaign.” Paper presented at the annual meetings of the Midwest Political Science Association. Kahn, Kim Fridkin and Patrick J. Kenney. 1999. “Do Negative Campaigns Mobilize or Suppress Turnout? Clarifying the Relationship between Negativity and Participation.” American Political Science Review 93 (4): 877-89. Kinder, Donald R. and Lynn M. Sanders. 1990. “Mimicking Political Debate with Survey Questions: The Case of White Opinion on Affirmative Action for Blacks.” Social Cognition 8 (1): 73-103. King, James D. and Jason B. McConnell. 2003. “The Effect of Negative Campaign Advertising on Vote Choice: Mediating Influence of Gender.” Social Science Quarterly 84 (4): 843-57. Lau, Richard R., Lee Sigelman, Caroline Heldman, and Paul Babbitt. 1999. “The Effects of Negative Political Advertisements: A Meta-Analytic Assessment.” American Political Science Review 93 (4): 851-76. Martin, Paul S. 2004. “Inside the Black Box of Negative Campaign Effects: Three Reasons Why Negative Campaigns Mobilize.” Political Psychology 25 (4): 545-62.
Obama N 77.9 47 24.8 34 Selected Control McCain Obama N 25.6 82.0 68 81.4 24.6 63 All Self-Selectors McCain Obama N 26.5 80.3 115 79.5 24.7 97 Treatment Effects McCain average (95% CI) selectors, non-selectors Obama average (95% CI) selectors, non-selectors -3.5 (-13.0, 6.0) -10.0, 1.0 4.2 (-5.1, 13.5) 8.8, 1.0 -2.1 (-9.6, 5.4) -5.0, -0.6 -5.5 (-16.2, 5.1) -9.4, -3.4 Figure 1. The Self-Selection Experiment and Heterogeneity Distributions and Expected Means for response variable Y: fY,SC 𝛼 selected control (n) self-selection random assignment p1 1-𝛼 fY,ST selected treatment (s)