SlowGuess/NAACL_2022_OCR huggingface dataset:
Individual differences encode error and preferences into choices allowing for consensus to arise, e.g., a person with low numeracy will sample interval utilities from a logistic distribution with a larger spread than someone with high numeracy who samples utility close to the true utility. Because the error encoded utilities are sampled, the preferred choice can change on different runs of the problem; however, individual differences influence the average choice. Figure 3 shows a snapshot when one parameter was altered while the others
were fixed and how these parameters can alter utilities to prefer certain choices across frames in the ADP.
Figure 3: Effects of individual differences on the ADP.
5 Learning Individual Differences
The last challenge we explore is how optimal individual-level parameters, NFC, NUM and RS can be inferred in GL-RDM and IL-RDM by optimising the following objective functions.
$$
\underset {i v d} {\arg \min } \sum_ {P, P} ^ {\operatorname {E x p}} \left| P _ {\text {g a i n}} - \hat {P} _ {\text {g a i n}} (i v d) \right| + \left| P _ {\text {l o s s}} - \hat {P} _ {\text {l o s s}} (i v d) \right| \tag {4}
$$
$$
\hat {P} _ {\text {g a i n}} (i v d) = \sum_ {e x p \in \operatorname {E x p}} \frac {\mathbb {E} [ \mathrm {C D M} (\mathrm {R D M P} _ {g a i n} , i v d) = \text {r i s k y} ]}{| \mathrm {E x p} |} \tag {5}
$$
GL-RDM Objective. Is given by equation 4 where  $ivd =$  (NUM, NFC, RS) are individual-level parameters which represent the characteristics of the entire group and  $\mathrm{Exp} = \{\mathrm{e}_1, \mathrm{e}_2, \dots, \mathrm{e}_i\}$  is the set of results based on human psychological experiments where each  $e$  is a 5-tuple described in the task formulation in section 2.2.
IL-RDM Objective. Is given by equation 6 where  $ivd_{i} =$  (NUM, NFC, RS) is the parameters which characterises individual  $i$ , RDP is a set of  $n$  RDMPs and  $PC$  is the mapping function of individuals to their preferred choices defined in the task formulation in section 2.2. Thus, the goal is to learn optimal individual parameters for each individual which maximises the expectation that CDM chooses their true preferred choice over all RDPs.
$$
\arg \max  _ {i v d} \sum_ {j} ^ {| \mathrm {R D P} |} \mathbb {E} [ \mathrm {C D M} (\mathrm {r d p} _ {j}, i v d _ {i}) = P C (i v d _ {i}, \mathrm {r d p} _ {j}) ] \tag {6}
$$
Categorical News. A dataset used for training/finetuning Cat2Vec and benchmark algorithms. The
dataset contains 22601 news articles with binary sentiments labelled from various news outlets dating from February to April 2020 using the Google News API spanning 46 news categories, e.g. travel, entertainment and death.
Group Risky Decision Making. A dataset of 88 psychological human experiments results grouped into categories used in the evaluation of the BR model. The categories represent differences in experiment controls and participants that undertook each experiment, e.g., 'ADP; within-subjects, low PISA'. The category outlines the risky decision-making problem; experimental design which can be grouped into within, where each participant is given both frames of a decision or between subject designs, where two independent groups answer each frame; and numeracy of participants, based on the performance of the country in which the experiment took place in the Program for International Student Assessment (PISA) (Stacey, 2015).
Individual 2-RDMP Prediction. A curated dataset of 38 unique 2-RDMPs selected from various psychological experiments regarding risky decision-making answered by 121 university students using a within-subject experimental design. Of the 38 2-RDMPs, most problems contain a corresponding gain and loss frame, e.g., the ADP in the introduction, each frame is considered a separate problem. Participants selected their preferred choice from the same pre-shuffled RDMP set<sup>1</sup> and no pre/post-processing of data was performed.
We apply different evaluation metrics suitable for each RDM task. For GL-RDM, we compare the true log-odds ratio (LOR) given by equation 7, between experimental results predicted by our model and the BR baseline model. Intuitively, the LOR measures the consistency of choices across the gain and loss frames.
$$
\operatorname {L O R} \left(P _ {\text {g a i n}}, P _ {\text {l o s s}}\right) = \ln \left(\frac {P _ {\text {g a i n}} \left(1 - P _ {\text {l o s s}}\right)}{P _ {\text {l o s s}} \left(1 - P _ {\text {g a i n}}\right)}\right) \tag {7}
$$
To determine the goodness-of-fit between the predicted LOR, we apply a hypothesis test, the Wald statistic  $(\chi^2)$  given by equation 9. The standard error (SE) is given by equation (8) where  $n_{\mathrm{safe,gain}}$  represents the number of individuals choosing the safe choice in the gain frame,
$n_{\mathrm{safe,loss}}$ ,  $n_{\mathrm{risky,gain}}$ ,  $n_{\mathrm{risky,loss}}$  can be derived similarly. The standard error asymptotically approaches a normal distribution when  $n$  is sufficiently large; thus, the associated Wald statistic, equation (9), follows a chi-square distribution with one degree of freedom.
$$
\mathrm {S E} = \sqrt {\frac {1}{n _ {\text {s a f e , g a i n}}} + \frac {1}{n _ {\text {s a f e , l o s s}}} + \frac {1}{n _ {\text {r i s k y , g a i n}}} + \frac {1}{n _ {\text {r i s k y , l o s s}}}} \tag {8}
$$
$$
\chi^ {2} = \left(\frac {\operatorname {L O R} \left(P _ {\text {g a i n}} , P _ {\text {l o s s}}\right) - \operatorname {L O R} \left(\hat {P} _ {\text {g a i n}} , \hat {P} _ {\text {l o s s}}\right)}{\mathrm {S E}}\right) ^ {2} \tag {9}
$$
To compare the parsimony and implicitly the error between our the BR and null (Busemeyer et al., 2015) models, we use the Akaike information criterion (AIC) and Bayesian information criterion (BIC). For IL-RDM, we evaluate the accuracy of each model correctly predicting the true choices for each individual.
6.3 Benchmark Algorithms
In the paper, we use two different sets of baselines. For GL-RDM, we directly compare our model against the BR model. Due to the small number of experiments per grouping in the GL-RDM dataset, to maintain parity with the BR baseline, we apply the same jackknife-leave-one-out (JLOO) method used for parameter estimation in the BR baseline model to avoid post-hoc parameter estimation (Busemeyer and Wang, 2000). Formally, given  $m$  observed human risky decision making experiment results within a category of comparable RDPs,  $\mathrm{Exp} = \{\mathrm{e}_1,\mathrm{e}_2,\dots ,\mathrm{e}_m\}$  as described in the task formulation in section 2.2. We wish to estimate  $m$  values of  $G_{i} = (\mathrm{NUM}_{i},\mathrm{NFC}_{i},\mathrm{RS}_{i})$  where  $G_{i}$  is group-level differences relating to observed experiments  $i$  where  $i = 1\dots m$ . To achieve this, we apply the  $G_{i}$  can be estimated by equation (10) where  $\mathrm{ER}_{-i}$  is the set of experimental results excluding  $\mathrm{er}_i$  as not to use the result in the estimation.
$$
\arg \min  _ {G _ {i}} \sum_ {P _ {\text {g a i n}}, P _ {\text {l o s s}}} ^ {\mathrm {E R} _ {- i}} | P _ {\text {g a i n}} - \hat {P} _ {\text {g a i n}} (G _ {i}) | + | P _ {\text {l o s s}} - \hat {P} _ {\text {l o s s}} (G _ {i}) | \tag {10}
$$
For IL-RDM, due to a lack of existing benchmark algorithms we compare our model against two baselines (1) Naive binary model using pretrained transformer language models where all
See appendix, section A.3 for questionnaire
RDM-choices are combined as a single input and outputs 0 or 1 corresponding to the safe or risky choice. (2) Sentiment analysis models as claim C2 asserts sentiments are highly influential in decision-making where decisions are based on choices with the highest positive sentiment. Random: Uniformly samples one of the available choices. Vader: A rule-based sentiment analysis for social media(Gilbert and Hutto, 2014). XLNet: SOTA pretrained autoregressive language model fine-tuned on the news dataset sentiments (Yang et al., 2019). ULMFiT: Pretrained language model fine-tuned on the news dataset sentiments using inductive transfer learning (Howard and Ruder, 2018).
6.4 Experiment Results
GL-RDM results. Table 1 (full table A.4 in Appendix) shows key discrepancies between our computational model, CDM, compared to both the actual LOR based on all 88 human experiments and those predicted by the BR model. Within each category, we find optimal group-level parameters which minimise (4) to calculate the predicted LOR of our model for each experiment using the jackknife-leave-one-out (JLOO) method to maintain comparability between the BR model.
Reference	Actual	BR	CDM	SE	x2
Standard ADP; one presentation, between-subjects, low PISA
(1) Tversky and Kahneman (1981)	2.20	1.65*	1.86	.26	1.83
(2) Mayhorn et al. (2002), Young	2.98	1.68	1.50	.58	5.89*
TOTAL of 14 predicted: 13 (93%)
Standard ADP; within-subjects, low PISA
(3) LeBoeuf and Shafir (2003) Exp #2	.57	1.05*	.81	.17	.52
TOTAL of 3 predicted: 3 (100%)
Standard ADP; multiple presentations, between-subjects, low PISA,
(4) Jou et al. (1996)	2.01	.87*	1.13	.33	6.99*
TOTAL of 6 predicted: 5 (83%)
Other problems; multiple presentations, between-subjects, high PISA
(5) Kühberger (1995) Plant #2	2.34	.7*	.34	.73	7.50*
TOTAL of 4 predicted: 3 (75%)
Zero-complement problems; multiple presentations, between-subjects
(6) Kühberger and Tanner (2010) Crops	-.43	0	.34	.30	6.83*
(7) Kühberger and Tanner (2010) Fish disease	.83	0*	.37	.30	15.74*
TOTAL of 7 predicted: 5 (71.4%)
“400 not saved” certain-option problems; multiple presentations, between-subjects, high PISA
(8) Kühberger (1995) Plant #1	.49	-.88*	.11	.58	.50
(9) Kühberger (1995) Cancer #1	-1.36	-.74	.11	.60	6.21*
TOTAL of 5 predicted: 6 (83.3%)
OVERALL TOTAL of 88 predicted: 82 (93.2%)
* Indicates results with Wald statistics over 1 degree of freedom
Note: Bolded rows indicate results where CDM outperforms or is equal to the BR model
Table 1: Group Level Experiment Results.
Critically, our results show that our computational model is capable of automating the prediction of human risky decision making on a wide range of RDMPs by predicting 82 of 88 (93.2%) experiments based on the Wald statistic. These results hold even when RDMPs were manipulated to capture a wider gamut of decision making through variations on framing and truncation of choices
where options were removed (Reyna et al., 2014). These results are comparable to carefully crafted human conducted analysis using the BR model which also predicted 82 of 88 experiments.
To further demonstrate the parsimony of our model compared to the BR and null models by applying the AIC and BIC metrics under a null CDM model where parameters are set to 0, we get AIC=14941 and BIC=14950. Whereas under the null model of the BR model, AIC=14981 and BIC=14986. In the best cases, our model outperforms all variations of the BR baseline model, with our model attaining AIC=13374 and BIC=13383 compared to AIC=13409 and BIC=13510. Furthermore, taking into consideration the relative likelihood ratio (RLR) to compare models using the AIC scores,  $\exp((13374 - 13409)/2) = 2.5 \times 10^{-8}$ , yields a significant result where the BR baseline model is only  $2.5 \times 10^{-8}$  times as probable as our model to minimize the information loss. Thus, our model attains better goodness-of-fit compared to the BR model while using significantly fewer parameters, 3 compared to up to 172 in the BR model.
IL-RDM results. Table 2 displays the average 5-fold cross-validation result predicting all 121 individuals' decisions for all 38 questions. Our model with a modest  $63.19\%$  accuracy outperforms all benchmark algorithms which hover around  $50\%$  for sentiment and  $60\%$  for pretrained language model baselines. This reinforces that IL-RDM is a more challenging problem and although sentiment analysis is important for decision-making, current SOTA sentiment analysis is not suitable for IL-RDM and only performs comparably to random choice. It is worth noting that while pretrained language models can be naively applied to IL-RDM with competitive results, they can not be naively applied to GL-RDM which requires the simultaneous predictions of two distributions of choices across frames where often the same RDM and choices is used across all experiments within a category.
Also displayed in the table are results when using transformers as encoders within Cat2Vec and results from a minor ablation study. For IL-RDM, transformers do not significantly improve accuracy as the resulting predicted categories and sentiments of categories from RDM-choices are highly similar between encoders. In the ablation study where choices are derived based on preferred choices at different levels of representation, i.e.,  $\mathrm{CDM}_{\mathrm{Categorical}}$  and  $\mathrm{CDM}_{\mathrm{Interval}}$ , reinforces the full
expressiveness of our model comes from the consensus between levels of representation and influence of individual differences.
Algorithm	Accuracy (%)	Min (%)	Max (%)
Random	49.87 ± 0.4	48.99	51.23
Vader	51.37 ± 0.9	48.64	54.33
XLNet	49.83 ± 2.1	44.21	56.31
ULMFiT	47.08 ± 2.1	39.31	53.30
BERT	61.94 ± 0.7	60.80	64.70
GTP2	53.65 ± 1.1	51.44	57.95
roBERTa	60.83 ± 1.7	57.23	65.87
CDM (Bi-LSTM)	62.47 ± 0.6	60.43	64.11
CDM (BERT)	62.30 ± 1.2	59.54	65.87
CDM (GTP2)	63.19 ± 1.2	60.64	68.00
CDM (roBERTa)	62.68 ± 1.2	59.43	66.35
Sentiment Cat2Vec	53.72 ± 1.4	48.04	56.67
CDMCategorical	51.86 ± 1.3	46.38	54.33
CDMInterval	53.17 ± 1.2	48.40	55.84
Table 2: Individual-Level Experiment Results
6.5 Error Analysis and Discussion
To understand the shortfall of our model for both GL-RDM and IL-RDM, we analyse cases in which our model fails to predict human decision-making. In GL-RDM, of the 6 experiments that our model did not successfully predict, 3 of these ((4),(5) and (7) in table 1) were not predicted by the BR baseline model indicating problems with parameter estimation using JLOO as these experiments are outliers with relatively significant differences in LORs within their respective categories.
In IL-RDM, inconsistencies exist across 2-RDMPs due in part to the within-subject design as participants may notice the underlying problem causing them to compare between problems rather than independently (Kahneman and Frederick, 2002). For example, figure 4 shows loss frames where individuals overwhelming preferred the safe choice, e.g., Q2, Q4 and most gain-loss pairs do not show a clear distinction between safe and risky choices in opposing frames, e.g. (Q26, Q2). Both cases are inconsistent with psychological studies.
Figure 4: Ratio of choices for all questions with gain and loss frames grouped.
Quantity of data is also an issue. While the number of participants and questions answered were quite large for psychology experiments, this dataset is relatively small for machine learning tasks. Figure 5(A) shows the histogram of all 121 individuals' test accuracy on one fold. Overall, our model can predict most individual's choices accurately, but the average is lowered by some individuals our model cannot predict due to inconsistencies mentioned in (1). Also, since the size of each fold is relatively small, containing 7-8 test RDPs, any RDP not predicted correctly will cause a large decrease in accuracy. Figure 5(B) shows the percentage of correct choices our model predicts for all individuals from the combined 5-fold test questions. On average, our model predicts gain and loss RDMPs relatively equally with accuracy of  $65.73\%$  and  $62.84\%$  respectively. However, RDMPs with "both frame", contains choices with combined gain and loss wording, cannot be predicted by our model due to this duality with an average of  $42.73\%$ .
Figure 5: (A) Histogram of individual accuracy on one fold and (B) combined k-fold test accuracy per question
This paper provides the first steps into a fully computational framework of risky decision-making, which adopts the cognitive and psychological basis of FTT with our model outperforming baselines in individual and group RDP prediction. Potential applications of our model are wide-ranging for scenarios in which predicting and understanding the characteristics of human risky decision-making is pivotal, e.g., the design of safety mechanisms based on how people make decisions in risky scenarios or in improving personalised recommendation systems based on understanding the users' personal traits and how they make decisions. Future work, therefore, involves adapting our model towards real-world applications, exploration of generalised decision-making and the design and evaluation of sophisticated end-to-end machine learning models for text-based decision-making.
Maurice Allais. 1953. Le comportement de l'homme rationnel devant le risque: critique des postulats et axiomes de l'école americaine. *Econometrica: Journal of the Econometric Society*, pages 503-546.
Charles J Brainerd and Valerie F Reyna. 1990. Gist is the grist: Fuzzy-trace theory and the new intuitionism. Developmental Review, 10(1):3-47.
David Broniatowski and Valerie Reyna. 2014. A mathematical formalization of fuzzy trace theory. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 36.
David A Broniatowski and Valerie F Reyna. 2018. A formal model of fuzzy-trace theory: Variations on framing effects and the allais paradox. Decision, 5(4):205.
Jerome Busemeyer, Zheng Wang, and Richard Shiffrin. 2015. Bayesian model comparison favors quantum over standard decision theory account of dynamic inconsistency. Decision, 2:1-12.
Jerome R Busemeyer and Yi-Min Wang. 2000. Model comparisons and model selections based on generalization criterion methodology. Journal of Mathematical Psychology, 44(1):171-189.
John T Cacioppo, Richard E Petty, Jeffrey A Feinstein, and W Blair G Jarvis. 1996. Dispositional differences in cognitive motivation: The life and times of individuals varying in need for cognition. *Psychological bulletin*, 119(2):197.
Paul S Carlin. 1990. Is the allais paradox robust to a seemingly trivial change of frame? Economics Letters, 34(3):241-244.
Li Chen, Marco De Gemmis, Alexander Felfernig, Pasquale Lops, Francesco Ricci, and Giovanni Semeraro. 2013. Human decision making and recommender systems. ACM Transactions on Interactive Intelligent Systems (TiiS), 3(3):1-7.
John Conlisk. 1989. Three variants on the allais example. The American Economic Review, pages 392-407.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
James N Druckman. 2001a. Evaluating framing effects. Journal of economic psychology, 22(1):91-101.
James N Druckman. 2001b. Using credible advice to overcome framing effects. Journal of Law, Economics, and Organization, 17(1):62-82.
James N. Druckman and Rose McDermott. 2008. Emotion and the framing of risky choice. Political Behavior, 30(3):297-321.
Ward Edwards. 1954. The theory of decision making. Psychological bulletin, 51(4):380.
N. Fagley and Paul Miller. 1990. The effect of framing on choice: Interactions with risk taking propensity, cognitive style, and sex. *Personality and Social Psychology Bulletin*, 16:496-510.
Peter Fischer, Eva Jonas, Dieter Frey, and Andreas Kastenmüller. 2008. Selective exposure and decision framing: The impact of gain and loss framing on confirmatory information search after decisions. Journal of Experimental Social Psychology, 44(2):312-320.
Milton Friedman and Leonard J Savage. 1952. The expected-utility hypothesis and the measurability of utility. Journal of Political Economy, 60(6):463-474.
CHE Gilbert and Erric Hutto. 2014. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In *Eighth International Conference on Weblogs and Social Media* (ICWSM-14). Available at (20/04/16) http://comp.social.gatech.edu/papers/icwsm14.vader.hutto.pdf, volume 81, page 82.
Faruk Gul, Wolfgang Pesendorfer, et al. 2008. Measurable ambiguity. Levine's Bibliography.
Thorvald Haerem, Bård Kuvaas, Bjørn T Bakken, and Tone Karlsen. 2011. Do military decision makers behave as predicted by prospect theory? Journal of Behavioral Decision Making, 24(5):482-497.
Scott Highbouse and Payam Yüce. 1996. Perspectives, perceptions, and risk-taking behavior. Organizational Behavior and Human Decision Processes, 65(2):159-167.
D Sunshine Hillygus and Todd G Shields. 2005. Moral issues and voter decision making in the 2004 presidential election. PS: Political Science and Politics, 38(2):201-209.
Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146.
Steffen Huck and Wieland Müller. 2012. Allais for all: Revisiting the paradox in a large representative sample. Journal of Risk and Uncertainty, 44(3):261-293.
Jerwen Jou, James Shanteau, and Richard Harris. 1996. An information processing view of framing effects: The role of causal schemas in decision making. Memory cognition, 24:1-15.
Alex Kacelnik and Melissa Bateson. 1997. Risk-sensitivity: crossroads for theories of decision-making. Trends in cognitive sciences, 1(8):304-309.
Daniel Kahneman. 2003. Maps of bounded rationality: Psychology for behavioral economics. American economic review, 93(5):1449-1475.
Daniel Kahneman and Shane Frederick. 2002. Representativeness revisited: Attribute substitution in intuitive judgment. Heuristics and biases: The psychology of intuitive judgment, 49:81.
Daniel Kahneman and Amos Tversky. 1980. Prospect theory. *Econometrica*, 12.
Anton Kühberger. 1995. The framing of decisions: A new look at old problems. Organizational Behavior and Human Decision Processes, 62(2):230-240.
Anton Kühberger and Patricia Gradl. 2013. Choice, rating, and ranking: Framing effects with different response modes. Journal of Behavioral Decision Making, 26(2):109-117.
Anton Kühberger and Carmen Tanner. 2010. Risky choice framing: Task versions and a comparison of prospect theory and fuzzy-trace theory. Journal of behavioral decision making, 23(3):314-329.
Anton Kühberger. 1995. The framing of decisions: A new look at old problems. Organizational Behavior and Human Decision Processes, 62(2):230-240.
Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In International conference on machine learning, pages 1188-1196.
Robyn LeBoeuf and Eldar Shafir. 2003. Deep thoughts and shallow frames: On the susceptibility to framing effects. Journal of Behavioral Decision Making, 16:77-92.
Jennifer S Lerner, Ye Li, Piercarlo Valdesolo, and Karim S Kassam. 2015. Emotion and decision making. Annual review of psychology, 66:799-823.
Irwin Levin, Gary Gaeth, Judy Schreiber, and Marco Lauriola. 2002. A new look at framing effects: Distribution of effect sizes, individual differences, and independence of types of effects. Organizational Behavior and Human Decision Processes, 88:411-429.
Zhiyuan Liu, Yankai Lin, and Maosong Sun. 2020. Representation learning and nlp. In Representation Learning for Natural Language Processing, pages 1-11. Springer.
David R Mandel. 2001. Gain-loss framing and choice: Separating outcome formulations from descriptor formulations. Organizational Behavior and Human Decision Processes, 85(1):56-76.
David R Mandel. 2014. Do framing effects reveal irrational choice? Journal of Experimental Psychology: General, 143(3):1185.
Christopher B Mayhorn, Arthur D Fisk, and Justin D Whittle. 2002. Decisions, decisions: Analysis of age, cohort, and time of testing on framing of risky decision options. Human Factors, 44(4):515-521.
Daniel McFadden. 2001. Economic choices. American economic review, 91(3):351-378.