NBER WORKING PAPER SERIES CORRELATION NEGLECT IN STUDENT TO SCHOOL MATCHING
Year: 2020

A growing body of evidence suggests that decision makers fail to account for correlation in signals that they receive. We study the relevance of this mistake in students' interactions with school choice matching mechanisms. In a lab experiment presenting simple and incentivized school choice scenarios, we find that subjects tend to follow optimal application strategies when schools' admissions decisions are determined independently. However, when schools rely on a common priority inducing correlation in admissions decision making suffers: application strategies become substantially more aggressive and fail to include attractive "safety" options. We document that this pattern holds even within subject, with significant fractions of participants applying to different programs when correlation is varied but all payoff relevant elements are held constant. We provide a battery of tests suggesting that this phenomenon is at least partially driven by correlation neglect, and we discuss implications that arise for the design and deployment of student to school matching mechanisms. Alex Rees Jones Cornell University Economics Department 404 Uris Hall Ithaca, NY 14850 and NBER arr34@cornell.edu Ran Shorrer Department of Economics The Pennsylvania State University Kern Building University Park, PA 16802 rshorrer@gmail.com Chloe J. Tergiman The Pennsylvania State University 334 Business Building University Park, PA 16803 chloejt@gmail.com A data appendix is available at http:/www.nber.org/data appendix/w26734 2 CORRELATION NEGLECT IN STUDENT TO SCHOOL MATCHING A growing body of evidence suggests that many people struggle with decision making in the presence of correlation. In typical examples of this problem, decision makers are presented with multiple signals that are each influenced both by independent components and information from a common source. The process by which signals are generated induces correlation, and optimal decision making requires taking it into account. In practice, however, experiments like those of Enke and Zimmermann (2019) demonstrate that many decision makers neglect to do so, effectively acting as if these correlated signals are independent. In this paper, we study the prevalence and consequences of these types of failures of reasoning in a decision of considerable importance: the application strategies of students applying to schools. Many application processes inherently require students to make forecasts of events determined by common underlying inputs, resulting in correlation structures like those described above. For example, students commonly must whittle a large number of schools down to a smaller set that are applied to or ranked, introducing an incentive to avoid listing two programs with highly correlated admissions decisions. In such environments, a student harboring correlation neglect faces a challenging decision. To illustrate these difficulties, consider a simple example. Imagine there are three programs at which you could potentially match, offering payoffs of 3, 2, and 1. Call these programs the best, middle, and worst programs, respectively. These programs will all rank you based on a common, currently unknown, priority score; assume it will be a random integer drawn from a uniform distribution ranging from 0 to 99. The best program will admit you if your score is at least 50. The middle program will admit you if your score is at least 45. The worst program will admit you with any score. If you could only apply to two of these programs, to which two would you apply? When considering this problem, one might feel the temptation to apply to the two programs with the highest payouts we will refer to this as the aggressive application strategy. However, doing so is costly in expectation. Because these programs rely on the same score and have near identical thresholds, the probability of being accepted by the middle program conditional on being rejected by the best program is quite low (10% and insufficient to motivate a risk neutral decision maker from taking the sure payoff offered by applying to the worst school. Expected payout is maximized by applying to the best and the worst programs we will refer to this as the diversified application strategy. Consider next a slightly modified example. Imagine you are considering the same three programs, but now these programs rank you based on program specific, statistically independent priority scores. Again, these evaluations are drawn from a uniform distribution ranging from 0 to 99. The best program's score threshold remains at 50, and the worst program continues to admit anyone. However, the middle program's acceptance threshold is changed to 90. In this situation, to which two schools would you apply? 1For a recent discussion of optimal diversification strategies in these environments and their significant complexity see Shorrer (2019) CORRELATION NEGLECT IN STUDENT TO SCHOOL MATCHING 3 As above, applying to the best and the worst programs remains the expected value maximizing strategy. Moreover, the consequences of pursuing either the diversified or aggressive application strategies are exactly the same as in the first example. The diversified application strategy grants a 50% chance of enrollment at the best program and a 50% chance of enrollment at the worst program. The aggressive application strategy grants a 50% chance of enrollment at the best program and, conditional on rejection there, a 10% chance of enrollment at the middle program. If one is restricted to these two strategies, choices across these scenarios should be identical. As we document in this paper, students' application strategies across these scenarios are quite different. When outcomes are correlated, a substantial fraction of students apply to the two most selective programs i.e. they apply aggressively. By contrast, when priorities are determined independently, subjects' intelligently pursue the diversified application strategy at a much higher rate. Despite the numerical equivalency of probabilities, subjects act as if a 10% conditional probability of acceptance (the relevant probability for decision making in the first example) is much more appealing than a transparent 10% unconditional probability of acceptance (the relevant probability for decision making in the second example) These simple examples illustrate something we believe to be a pervasive feature of school choice. In many environments, students can only apply to a subset of the schools that they see as attractive. In such situations, correlation in evaluations at different programs may be neglected or underweighted, leading students to fail to apply the appropriate contingent reasoning when deciding whether to apply to programs of similar selectivity. The consequence is inadequate diversification of application portfolios conceptually similar to the inadequate diversification of asset holdings that is attributed to correlation neglect in Eyster and Weizsacker (2016) Concern about decision quality in the face of correlated admissions is more than academic. As we summarize in Section 1, many countries use school assignment systems that involve choices much like the scenario just considered. In these systems, students are required to submit constrained lists of applications before discovering the results of the common test used to determine priority. To the extent that students are unable to correctly reason in such environments, intervention and revision of these systems may be merited. To study this issue systematically, we ran a laboratory experiment among 165 students of Penn State University in early 2019. Subjects faced incentivized application scenarios much like the example above. In each scenario, subjects provided a rank order list (ROL) to be used to match them to one of three schools. These ROLs could only contain two items, however, and thus required the student to choose a school application to forego. To study the role of correlation neglect, we generate both within subject and between subjects variation in the correlation of admissions decisions. The presence or absence of correlation was 2Furthermore, while we have emphasized expected value maximizing behavior in our example, this equivalence is expected to hold more broadly. Indeed, it should hold so long as preferences depend only on the induced probability distribution over final matches. 3For detailed experimental examination of the difficulties of contingent reasoning in the presence of uncertainty, see Martinez Marquina et al. (2017) 4 CORRELATION NEGLECT IN STUDENT TO SCHOOL MATCHING governed by whether programs' priorities were determined by either a single, common priority score or by program specific, independent priorities, respectively. Subjects began the experiment by completing nine scenarios across which the acceptance thresholds varied but with the same correlation condition. They then completed a second battery of the nine scenarios presented under the other correlation condition. Comparing choices across these conditions provides a clean between subjects analysis of how application strategies respond to correlated evaluation. A perhaps stronger test arises from the within subject examination of "matched pairs" of scenarios, in which given strategies result in the same distribution of outcomes in both conditions. Across these scenarios we document a systematic and quantitatively large tendency to fail to pursue the diversified strategy in the presence of correlation, in effect foregoing desirable "safety" options. We present two classes of evidence suggesting that this relates to incorrect processing of correlated environments. First, to provide a benchmark for correct processing, we present subjects with direct choices over monetary gambles. These monetary gambles were constructed to match the gamble induced by different application strategies in the scenarios seen by the subject. We find that choices i