SlowGuess/NAACL_2022_OCR huggingface dataset:
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
Paul Miller and N. Fagley. 1991. The effects of framing, problem variations, and providing rationale on choice. *Personality and Social Psychology Bulletin*, 17:517-522.
David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Linguisticae Investigationes, 30(1):3-26.
Hidetaka Okder. 2012. The illusion of the framing effect in risky decision making. Journal of Behavioral Decision Making, 25(1):63-73.
Valerie F Reyna and Charles J Brainerd. 1991. Fuzzytrace theory and framing effects in choice: Gist extraction, truncation, and conversion. Journal of Behavioral Decision Making, 4(4):249-262.
Valerie F Reyna, Christina F Chick, Jonathan C Corbin, and Andrew N Hsia. 2014. Developmental reversals in risky decision making: Intelligence agents show larger decision biases than college students. *Psychological science*, 25(1):76-84.
Michael Ronnlund, Erik Karlsson, Erica Laggnäs, Lisa Larsson, and Therese Lindström. 2005. Risky decision making across three arenas of choice: Are younger and older adults differently susceptible to framing effects? The Journal of general psychology, 132:81-92.
Uzi Segal. 1987. The ellsberg paradox and risk aversion: An anticipated utility approach. International Economic Review, pages 175-202.
Herbert A Simon. 1993. Decision making: Rational, nonrational, and irrational. Educational Administration Quarterly, 29(3):392-411.
Kaye Stacey. 2015. The international assessment of mathematical literacy: Pisa 2012 framework and items. In Selected regular lectures from the 12th International Congress on Mathematical Education, pages 771-790. Springer.
Keith E. Stanovich and Richard F. West. 1998. Individual differences in framing and conjunction effects. Thinking & Reasoning, 4(4):289-317.
Jeannette M Stein. 2012. Framing effects: The influence of handedness and access to right hemisphere processing. *Laterality: Asymmetries of Body, Brain and Cognition*, 17(1):98-110.
Stanley Smith Stevens et al. 1946. On the theory of scales of measurement.
Kazuhisa Takemura. 1994. Influence of elaboration on the framing of decision. The Journal of Psychology, 128(1):33-39.
R Scott Tindale, Susan Sheffey, and Leslie A Scott. 1993. Framing and group decision-making: Do cognitive changes parallel preference changes? Organizational Behavior and Human Decision Processes, 55(3):470-485.
Amos Tversky and Daniel Kahneman. 1981. The framing of decisions and the psychology of choice. science, 211(4481):453-458.
Amos Tversky and Daniel Kahneman. 1992. Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and uncertainty, 5(4):297-323.
John Von Neumann and Oskar Morgenstern. 2007. Theory of games and economic behavior (commemorative edition). Princeton university press.
Xiao Tian Wang. 1996. Framing effects: Dynamics and task domains. Organizational behavior and human decision processes, 68(2):145-157.
Xiao-Tian Wang and Victor S Johnston. 1995. Perceived social context and risk preference: A reexamination of framing effects in a life-death decision problem. Journal of Behavioral Decision Making, 8(4):279-293.
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. arXiv preprint arXiv:1906.08237.
Milan Zeleny. 2012. Multiple criteria decision making Kyoto 1975, volume 123. Springer Science & Business Media.
Yin-ling Zhang, Li-jun Xiao, Yu Ma, and Dan-min Miao. 2008. Effect of framing on risky choice: One case study in china. Social Behavior and Personality: an international journal, 36(5):651-658.
Yinling Zhang and Danmin Miao. 2008. Social cues and framing effects in risky decisions among chinese military students. Asian Journal of Social Psychology, 11(3):241-246.
A.1 Final Decision Algorithm
Algorithm 1 corresponds to the algorithm mentioned in section 6.1 of the main paper.
A.2 Evaluation Metric Calculations
The calculations for the second type of evaluation metric we use to compare the parsimony of our model against baseline algorithms are the Akaike information criterion (AIC) and Bayesian information criterion (BIC) are given by equations (13) and (14) using the log-likelihood calculated by equations
Algorithm 1 Computational Decision Making
Input:  $n$  RDMP, NUM, NFC, RS
Output: Decision/Preferred Choice
1: for choice in n-RDMP do
2: if  $\mathrm{CU}(\mathrm{Pref}_{\mathrm{Cat}},\mathrm{NFC}) <   \mathrm{CU}(\mathrm{choice},\mathrm{NFC})$  then
3:  $\mathrm{Pref}_{\mathrm{Cat}} = \mathrm{choice}$
4: if  $\mathrm{IU}(\mathrm{Pref}_{\mathrm{Int}},\mathrm{NUM}) <   \mathrm{IU}(\mathrm{choice},\mathrm{NUM})$  then
5:  $\mathrm{Pref}_{\mathrm{Int}} = \mathrm{choice}$
6: if  $\mathrm{Pref}_{\mathrm{Cat}} = \mathrm{Pref}_{\mathrm{Int}}$  then
7: return PrefCat
8: else if  $\mathrm{Uniform}(0,1)\leq \mathrm{RiskSensitivity(RS)}$  then
9: return Riskiest Choice
11: return Safest Choice
and (12). In these equations  $n_{1,1}$  is the number of people who chose the first choice (safe choice) in the first problem (gain frame),  $p_{1,1}$  is the predicted proportion of subjects who chose the first choice (safe choice) in the first problem (gain frame), etc. For the AIC,  $k$  is the total number of parameters of our model, 3 which correspond to each individual difference and in BIC,  $n$  is the total number of data points, 176 to represent the gain and loss frames in the 88 human experiments.
$$
\begin{array}{l} \ln \left[ L \left(y _ {i}\right) \right] = n _ {1, 1} \ln p _ {1, 1} + n _ {1, 2} \ln p _ {1, 1 2} \\ + n _ {2, 1} \ln p _ {2, 1} + n _ {2, 2} \ln p _ {2, 2} \tag {11} \\ \end{array}
$$
$$
\ln [ L (y) ] = \sum_ {i} \ln [ L (y _ {i}) ] \tag {12}
$$
$$
\operatorname {A I C} = 2 k - 2 \ln [ L (y) ] \tag {13}
$$
$$
\operatorname {B I C} = k \ln (n) - 2 \ln [ L (y) ] \tag {14}
$$
To compare models using AIC, the relative likelihood ratio (RLR) given in equation 15 can be applied which compares the probability that the BR baseline model minimises the estimated information loss compared to our CDM model given that  $\mathrm{AIC}_{\mathrm{CDM}} \leq \mathrm{AIC}_{\mathrm{BR}}$  where  $\mathrm{AIC}_{\mathrm{BR}}$  and  $\mathrm{AIC}_{\mathrm{CDM}}$  are corresponding AIC scores of each model.
$$
\mathrm {R L R} = \exp \left(\frac {\mathrm {A I C} _ {\mathrm {C D M}} - \mathrm {A I C} _ {\mathrm {B R}}}{2}\right) \tag {15}
$$
A.3 Individual Level Questionnaire
Full inventory of all 36 questions used in the Individual 2-RDMP Prediction dataset:
Q1: Which of the following options do you prefer?
(a) A sure win of $30
(b) 80% chance to win $45
Q2: Imagine that 6000 pieces of precious paintings in a world-famous museum are accidentally exposed to a disastrous chemical pollution. Two alternative plans to rescue these art treasures have been proposed. Assume that the exact estimates of the consequences of the plans made by scientists are as follows:
(a) If plan A is adopted, 4000 pieces will be destroyed by the chemical pollution.
(b) If plan B is adopted, there is a one-third probability that none of these paintings will be destroyed, and two-thirds probability that all 6000 of these paintings will be destroyed.
Q3: A large car manufacturer has recently been hit with a number of economic difficulties and it appears as if three plants need to be closed and 6000 employees laid off. The vice-president of production has been exploring alternative ways to avoid this crisis and has developed two plans:
(a) Plan C: This plan will result in the loss of 2 plants and 4000 jobs.
(b) Plan D: This plan has a  $2/3$  probability of resulting in the loss of 3 plants and all 6000 jobs, but has a  $1/3$  probability of losing no plants and no jobs
Q4: Imagine you recieve a letter from the president of a subsidiary describing a dilemma concerning whether to fight an impending patent violation suit or settle out of court that reads: If we do not agree to this proposal, PMG will file their suit. Going to court would involve the possibility of losing \(1,100,000 in damages and losing the Duraplast line. If we win in court, we will incur a small sum for legal expenses. Our corporate lawyer, Mr. Bell, and our outside law firm estimate that we have a 2 in 3 chance of losing the case.
(a) Agree to the proposal (no lawsuit)
(b) Disagree to the proposal: 2/3 chance of losing the lawsuit and incurring costs of $1100000
Q5: Imagine that you have lung cancer and you must choose between two therapies: surgery and radiation. Surgery for lung cancer involves an operation on the lungs. Most patients are in the hospital for two or three weeks and have some pain around their incisions; they spend a month or so recuper
ating at home. After that, they generally feel fine. Radiation therapy for lung cancer involves the use of radiation to kill the tumor and requires coming to the hospital about four times a week for six weeks. Each treatment takes a few minutes and during the treatment, patients lie on a table as if they were having an x-ray. During the course of the treatment, some patients develop nausea and vomiting, but by the end of the six weeks they also generally feel fine. Thus, after the initial six or so weeks, patients treated with either surgery or radiation therapy feel about the same.
(a) Surgery: Of 100 people having surgery, 90 live through the postoperative period, 68 are alive at the end of one year and 34 are alive at the end of five years.
(b) Radiation Therapy: Of 100 people having radiation therapy, all live through treatment, 77 are alive at the end of one year and 22 are alive at the end of five years.
Q6: Imagine that you brought $6000 worth of stock from a company that has just filed a claim for bankruptcy recently. The company now provides you with two alternatives to recover some of your money.
(a) You will save $2000 of your money
 (b) You will take part in a random drawing procedure with exactly a one-third probability of saving all $6000 of your money, and two-thirds probability of saving none of your money.
Q7: Imagine that in one particular state it is projected that 1000 students will dropout of school during the year, two programs have been prosed to address this problem, but only one can be implemented. Based on other states experiences with programs, estimates of the outcomes that can be expected for each program can be made.
(a) Program 1: 600 of the 1000 students will drop out of school
(b) Program 2: 2/5 chance that none of the 1000 students will drop out of school and 3/5 chance that all 1000 students will drop out of school
Q8: Assume that you have just been given a gift of $1000.
(a) Taking an additional $500 for sure.
 (b) Flipping a coin and winning another $1000
if heads comes up or getting no additional money if tails comes up.
Q9(i): Imagine that you face the following pair of concurrent decisions. First examine both decisions, then indicate the options you prefer.
(a) A sure gain of $240
(b)  $25\%$  chance to gain $1000, and  $75\%$  chance to gain nothing
Q9(ii): Imagine that you face the following pair of concurrent decisions. First examine both decisions, then indicate the options you prefer.
(a) A sure loss of $750
(b)  $75\%$  chance to lose $1000, and  $25\%$  chance to lose nothing
Q9(iii): Imagine that you face the following pair of concurrent decisions. First examine both decisions, then indicate the options you prefer.
(a)  $25\%$  chance to win $240, and  $75\%$  chance to lose $760
 (b) 25% chance to win $250, and 75% chance to lose $750
Q10: You are staying in a hotel room on vacation. You paid \(6.95 to see a movie on pay TV. After 5 minutes you are bored and the movie seems pretty bad. Would you continue to watch the movie or not?
(a) Continue to watch
 (b) Turn it off and lose $6.95
Q11: Imagine that your country is preparing for the outbreak of an unusual disease, which is expected to kill 600 people. Two alternative programs to combat the disease have been proposed. Assume that the exact scientific estimate of the consequences of the programs are as follows:
(a) If Program A is adopted, 200 people will be saved
(b) If Program B is adopted, there is 1/3 probability that 600 people will be saved, and 2/3 probability that no people will be saved
Q12: Imagine that you have decided to see a play where admission is  $10 per ticket. As you enter the theatre you discover that you have lost a$ 10 bill.
(a) Still pay $10 for a ticket for the play
 (b) Don't pay $10 for a ticket for the play
Q13: Consider the following two stage game. In the first stage, there is a  $75 \%$ chance to end the game without winning anything, and a  $25 \%$ chance to move into the second stage. If you reach the second stage, you have a choice between: A sure win of $30 and  $80 \%$ chance to win $45
(a) A sure win of $30
 (b) 80% chance to win $45
Q14: Imagine that six people in your family, including both of your parents, your brothers and your sisters, are infected by a fatal disease. Two alternative medical plans to treat the disease have been proposed. Assume that the exact scientific estimates of the consequences of the plans are as follows:
(a) If plan A is adopted, two of them will be saved.
(b) If plan B is adopted, there is a one-third probability that all six of them will be saved, and two-thirds probability that none of them will be saved.
Q15: Your are presented with the following report from the head of a special team assigned to investigate the prospects of a project in Arizona: Our new analysis indicates that, if we choose to compete with ATC, we would face the possibility of capturing only a small market share. This would give us an after-tax return on investment of as little as  $10\%$ , while capturing a large market share would give us a return of  $22\%$ . We estimate our chance of getting a small market share to be 2 in 3. If we were to team up with ATC on the terms proposed, our return would be  $14\%$  after tax, with the same total investment.
(a) Compete with ATC: 1/3 chance of gaining a large market share of  $22\%$  and 2/3 chance of gaining a small market share of  $10\%$
(b) Don't compete with ATC:  $100\%$  chance of capturing  $14\%$  market share
Q16: A committee found a fish disease in a nearby lake. About 12 fish species (among them the most popular dining fish) have the Proliferative Kidney Disease (PKD). This is a chronically developing infectious disease which can have deadly consequences for the fish. Young fish are especially susceptible, while others seem to be immune against an infection. Experts suggest that PKD is one cause of declining fish catches. The researchers
assume human activities and water pollution foster the spread of the disease. They are considering releasing more fish into the lake to control the epidemic. Imagine that you are a government official of the adjacent village.
(a) Option A: If the release of fish is implemented, 4 fish species will survive.
(b) Option B: If the release of fish is implemented, there is  $1/3$  probability that all of the 12 fish species will survive, and  $2/3$  probability that none of them will survive.
Q17: Imagine a refinery that processes petroleum products. An investigation found that due to tank leaks, both soil and drinking water became contaminated. Due to this contamination 720 children from the adjacent village have a fatal disease. There is agreement among experts that children will not suffer health problems, provided they have a strong immune system. Otherwise, it is likely that children will have serious health problems. A vaccine against this disease has been developed and tested. However, the vaccine sometimes can cause side effects that can be fatal too. You are an environmental activist with much influence on the local hospital and you have to decide if you want to lobby for the vaccination or not.
(a) Option C: If the vaccination is adopted, the health of 480 children will be damaged for sure.
(b) Option D: If the vaccination is adopted, there is a one-third probability that the health of none of the 720 children will be damaged, and a two-thirds probability that the health of all 720 of them will be damaged.
Q18: Assume that you have just been given a gift of $2000. But you now are forced to choose between the following two alternatives:
(a) Losing $500 for sure
 (b) Flipping a coin and losing $1000 if heads comes up or losing nothing if tails comes up
Q19: Which of the following options do you prefer?
(a)  $25\%$  chance to win $30
 (b) 20% chance to win $45
Q20: Imagine that you are about to purchase a
jacket for \(125, and a calculator for \)15. The calculator salesman informs you that the calculator you wish to buy is on sale for \(10 at the other branch of the store, located 20 minutes drive away.
(a) Make the trip to the other store and save 5 dollars but lose 20 minutes
(b) Don't make the trip to the other store and save 20 minutes but lose 5 dollars
Q21: Imagine that six people in your family, including both of your parents, your brothers and your sisters, are infected by a fatal disease. Two alternative medical plans to treat the disease have been proposed. Assume that the exact scientific estimates of the consequences of the plans are as follows:
(a) If plan A is adopted, four of them will die.
(b) If plan B is adopted, there is a one-third probability that none of them will die, and two-thirds probability that all six of them will die.
Q22: Imagine you recieve a letter from the president of a subsidiary describing a dilemma concerning whether to fight an impending patent violation suit or settle out of court that reads: If we do not agree to this proposal, PMG will file their suit. Going to court would involve the possibility of keeping the Duraplast line and incurring only a small sum for legal expenses. If we lose in court, we will incur $1,100,000 in damages. Our corporate lawyer, Mr. Bell, and our outside law firm agree that we have a 1 in 3 chance of winning the case.
(a) Agree to the proposal (no lawsuit)
(b) Disagree to the proposal: 1/3 chance of winning the case
Q23: Imagine that your country is preparing for the outbreak of an unusual disease, which is expected to kill 600 people. Two alternative programs to combat the disease have been proposed. Assume that the exact scientific estimate of the consequences of the programs are as follows:
(a) If Program C is adopted 400 people will die.
(b) If Program D is adopted there is 1/3 probability that no one will die, and 2/3 probability that 600 people will die.
Q24: You are staying in a hotel room on vacation.  
You turn on the TV and there is a movie on. After 5
minutes you are bored and the movie seems pretty bad. Would you continue to watch the movie or not?
(a) Continue to watch
(b) Turn it off
Q25: Imagine that six people are infected by a fatal disease. Two alternative medical plans to treat the disease have been proposed. Assume that the exact scientific estimates of the consequences of the plans are as follows:
(a) If plan A is adopted, four people will die.
(b) If plan B is adopted, there is a one-third probability that none of them will die, and two-thirds probability that all six people will die.
Q26: Imagine that 6000 pieces of precious paintings in a world-famous museum are accidentally exposed to a disastrous chemical pollution. Two alternative plans to rescue these art treasures have been proposed. Assume that the exact estimates of the consequences of the plans made by scientists are as follows:
(a) If plan A is adopted, 2000 pieces will be saved from the chemical pollution.
(b) If plan B is adopted, there is a one-third probability that all the 6000 paintings will be saved, and two-thirds probability that none of these paintings will be saved.
Q27: Imagine that in one particular state it is projected that 1000 students will dropout of school during the year, two programs have been prosed to address this problem, but only one can be implemented. Based on other states experiences with programs, estimates of the outcomes that can be expected for each program can be made.
(a) Program 1: 400 of the 1000 students will stay in school
(b) Program 2: 2/5 chance that all 1000 students will stay in school and 3/5 chance that none of the 1000 will stay in school
Q28: Imagine that you have lung cancer and you must choose between two therapies: surgery and radiation. Surgery for lung cancer involves an operation on the lungs. Most patients are in the hospital for two or three weeks and have some pain around their incisions; they spend a month or so recuperating at home. After that, they generally feel fine.
Radiation therapy for lung cancer involves the use of radiation to kill the tumor and requires coming to the hospital about four times a week for six weeks. Each treatment takes a few minutes and during the treatment, patients lie on a table as if they were having an x-ray. During the course of the treatment, some patients develop nausea and vomiting, but by the end of the six weeks they also generally feel fine. Thus, after the initial six or so weeks, patients treated with either surgery or radiation therapy feel about the same.
(a) Surgery: Of 100 people having surgery, 10 die during surgery or the postoperative period, 32 die by the end of one year and 66 die by the end of five years.
(b) Radiation Therapy: Of 100 people having radiation therapy, none die during treatment, 23 die by the end of one year and 78 die by the end of five years.
Q29: Imagine that you have decided to see a play and paid the admission price of $10 per ticket. As you enter the theatre you discover that you have lost the ticket. The seat was not marked and the ticket cannot be recovered.
(a) Pay $10 for another ticket
 (b) Don't pay $10 for another ticket
Q30: Your are presented with the following report from the head of a special team assigned to investigate the prospects of a project in Arizona: Our new analysis indicates that, if we choose to compete with ATC, we would have the possibility of capturing a large market share. This would give us an after-tax return on investment of as much as  $22\%$ , while capturing a small market share would give us a return of only  $10\%$ . We estimate a 1 in 3 chance of getting a large market share. If we were to team up with ATC on the terms proposed, our return would be  $14\%$  after tax, with the same total investment.
(a) Compete with ATC
(b) Don't compete with ATC
Q31: A large car manufacturer has recently been hit with a number of economic difficulties and it appears as if three plants need to be closed and 6000 employees laid off. The vice-president of production has been exploring alternative ways to avoid this crisis and has developed two plans:
(a) Plan A: This plan will save 1 plant and 2000
(b) Plan B: : This plan has a 1/3 probability of saving all 3 plants and all 6000 jobs, but has a  $2 / 3$  probability of saving no plants and no jobs
Q32: Imagine that you are about to purchase a jacket for \(15, and a calculator for \)125. The calculator salesman informs you that the calculator you wish to buy is on sale for \(120 at the other branch of the store, located 20 minutes drive away.
Q33: A committee found a fish disease in a nearby lake. About 12 fish species (among them the most popular dining fish) have the Proliferative Kidney Disease (PKD). This is a chronically developing infectious disease which can have deadly consequences for the fish. Young fish are especially susceptible, while others seem to be immune against an infection. Experts suggest that PKD is one cause of declining fish catches. The researchers assume human activities and water pollution foster the spread of the disease. They are considering releasing more fish into the lake to control the epidemic. Imagine that you are a government official of the adjacent village.
(a) Option C: If the release of fish is implemented, 8 fish species will die.
(b) Option D: If the release of fish is implemented, there is  $2/3$  probability that none of the 12 fish species will die, and  $1/3$  probability that all of the 12 fish species will die.