Abstract: Using Ratio Scaling to Assess Likelihood of Joint Events

Using Ratio Scaling to Assess Likelihood of Joint Events Han-Hui Por & David Budescu a Fordham University Published online: 02 Jun 2014. To cite this article: Han-Hui Por & David Budescu (2014) Abstract: Using Ratio Scaling to Assess Likelihood of Joint Events, Multivariate Behavioral Research, 49:3, 295-296, DOI: 10.1080/00273171.2014.912925 To link to this article: http://dx.doi.org/10.1080/00273171.2014.912925 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions Multivariate Behavioral Research, 49:295–296, 2014 Copyright C © Taylor & Francis Group, LLC ISSN: 0027-3171 print / 1532-7906 online DOI: 10.1080/00273171.2014.912925 Abstract: Using Ratio Scaling to Assess Likelihood of Joint Events Using Ratio Scaling to Assess Likelihood of Joint Events Han-Hui Por and David Budescu Fordham University Estimating the probabilities of uncertain events is a highly complex and demanding task. These estimates are often used to inform policies and decisions, which makes it critical that the estimates are of good quality. Traditionally, direct elicitations are used but extensive research on cognitive heuristics suggests the presence of bias in such direct, intuitive judgments of probabilities. In this paper we test a new elicitation method using pairwise comparisons of events and illustrate its use in the context of estimation of joint probabilities in 2 × 2 tables. Instead of making direct judgments, individuals compare pairs of possible events and identify which of the two is more likely and by how much. These pairwise estimates are aggregated to create a matrix of ratio judgments, R, where the ratios are scaled and from which target probabilities are extracted (Table 1). The investigation consisted of 2 experiments and a simulation study. In the first experiment, participants assessed the probabilities of events pairs, such as stock market gains This work was supported in part by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior National Business Center Contract D11PC20059 and in part by the Anastasi Fellowship at Fordham University. Correspondence concerning this abstract should be addressed toHan-Hui Por, Fordham University, 441 East Fordham Road, Psychology Department, Bronx, NY 10458. E-mail: hpor@fordham.edu and sunny days. They provided direct assessments of the events’ joint probabilities (e.g., probability of a stock market gain on a sunny day) and also made pairwise comparisons of the same joint events. Participants in the second experiment learned the distribution of hypothetical events pairs and provided direct and ratios estimates. The use of novel association in the second study leveled the playing field for all participants and provided objective metrics to determine accuracy. In both studies, the ratio estimates were significantly more accurate than the direct estimates. The superiority of the ratio-scaled estimates was also supported by the simulation study, which assumed that participants are accurate in their judgments but these judgments are perturbed by random error with variance σ 2. At the same values of σ 2 ratioscaled estimates are inherently more accurate than the direct ones. The pairwise comparison approach is a compelling and viable assessment method for eliciting subjective probabilities of joint events in 2 × 2 contingency tables. Because of its simplicity from the point of the user, the approach can be used as a means to extract probability estimates from nonexperts, harnessing the wisdom of the crowd to supplement experts’ judgments. In future research, the use of the pairwise approach can be expanded to estimate the probabilities of marginal and conditional events as well as parameters of prior distributions in Bayesian analysis. D ow nl oa de d by [ Fo rd ha m U ni ve rs ity ], [ M r D av id V . B ud es cu ] at 0 8: 08 0 5 Ju ne 2 01 4 296 SMEP STUDENT PAPER COMPETITION ABSTRACTS TABLE 1 Calculating the Joint Probability Estimates of Events A and B From Ratio Judgments