SlowGuess/NAACL_2022_OCR huggingface dataset:
Jaron Mar
jaron.mar@auckland.ac.nz
Jiamou Liu
jiamou.liu@auckland.ac.nz
Understanding, modelling and predicting human risky decision-making is challenging due to intrinsic individual differences and irrationality. Fuzzy trace theory (FTT) is a powerful paradigm that explains human decision-making by incorporating gists, i.e., fuzzy representations of information which capture only its quintessential meaning. Inspired by Broniatowski and Reyna's FTT cognitive model, we propose a computational framework which combines the effects of the underlying semantics and sentiments on text-based decision-making. In particular, we introduce Category-2-Vector to learn categorical gists and categorical sentiments, and demonstrate how our computational model can be optimised to predict risky decision-making in groups and individuals.
Imagine that your town is preparing for a viral outbreak which is projected to kill 600 people. Two alternative programs to combat the virus have been proposed. Assume that the exact scientific estimates of the consequences of the programs are Program A: “200 people will be saved”; and Program B: “1/3 probability that all 600 lives will be saved; 2/3 probability that no lives will be saved”. Given these choices, which program would you choose? Alternatively, if choices were presented as follows, which program would you choose? Program C: “400 people will die”; and Program D: “1/3 probability that no one will die and a 2/3 probability that all 600 will die”. This problem is a modified version of the Asian disease problem (ADP) (Tversky and Kahneman, 1981), a well-studied risky decision-making problem (RDMP) in psychology where decisions are made under risk or include probabilistic outcomes (Edwards, 1954). In this RDMP, programs A and B form the gain frame where choices are worded in a positive and optimistic manner, whereas programs C and D form
the loss frame where choices are written in a negative and pessimistic manner. Studies have validated that in the gain frame, humans overwhelmingly prefer the safe choice A (72%), whereas in the loss frame, they overwhelmingly prefer the risky choice D (78%) even though the choices and outcomes in both frames are equivalent (Tversky and Kahneman, 1981). This phenomenon, known as the Allais paradox, implies that observed human choices are inconsistent with predictions based on expected utility alone, thereby confirming the influence of language, wording of choices, and sentiments on human decision-making.
Being able to understand, model and predict human decision-making leads to many real-world applications, from predicting election results (Hillygus and Shields, 2005), to improving user experience in recommender systems (Chen et al., 2013). However, Allais paradox means that understanding the integral but complex cognitive process of decision-making, particularly in humans, is extremely challenging due to our diverse characteristics, beliefs and experiences. Furthermore, human decision-making is often fraught with irrationality even in the presence of overwhelming evidence against some choice or beliefs (Simon, 1993). This brings into question how human decision-making can be modelled with these complexities and nuances involved. This is an especially important task when considering current approaches to decision-making, such as utility theory which typically lacks any behavioural basis and ignore human sentiments during human decision-making (Lerner et al., 2015).
Our goal is to develop a model of automated human decision-making that bridges current decision-making techniques with fuzzy trace theory (FTT), an established cognitive theory to predict group and individual decision making outlined in sections 2.2. Originally proposed by Brainerd and Reyna in the 1990s, FTT aims to explain cognitive phenomena
Findings of the Association for Computational Linguistics: NAACL 2022, pages 391 - 409
in memory and reasoning (Brainerd and Reyna, 1990). In a nutshell, FTT posits that humans form two types of mental representations, known as verbatim which are detailed representations and gist which are fuzzy representations that only capture the most quintessential meanings, and people prefer to make decisions based on gist rather than verbatim representations.
In contrast with alternative cognitive and decision-making theories such as expected utility theory (Friedman and Savage, 1952) and prospect theory (Kahneman and Tversky, 1980), we adopt FTT for two reasons. Firstly, FTT is the most holistic cognitive model which encompasses theories of how information is stored in memory and how memory plays an important role in our decision-making rather than treating decision-making as an isolated process. Because of this, FTT provides us with an extensive set of tools to explain and evaluate decision-making. Secondly, is FTTs suitability for computational modelling as conceptual parallels can be drawn between representation learning, particularly in neural-based language modelling, and the process of creating gist representations by distilling the quintessential information. For example, popular embedding methods for words, sentences and documents in NLP aim to create fuzzy semantic representations through dimensionality reduction of language to semantic vectors which can be viewed as gist representations of the original language (Liu et al., 2020).
Contributions: We investigate two levels of text-based risky decision prediction tasks, group and individual-level prediction from a computational standpoint and incorporating state-of-the-art methods in NLP, we further investigate:
- How do gist representations of choices give rise to decisions? We present a framework of decision-making based on gist representation learning.
- How can we computationally encode gist representations based on the language of choices? We outline how gist representations can be computationally encoded using techniques in NLP and propose Category-to-Vector (Cat2Vec), to learn and predict categorical embeddings of choices.
- How can we extract the underlying sentiments of gist representations? By extending Cat2Vec, we show how sentiments can be learnt at a categorical level; this differs from traditional approaches
of sentiment analysis in NLP that examine sentiments at a text level.
- How can individual differences of individuals and groups be modelled, what impact do these differences have on decision-making? We propose that individual differences are mechanisms that can encode errors at various points in the decision-making process and propose an optimisation method to infer these individual differences.
- Finally, we demonstrate in experiments that our proposed model achieves state-of-the-art performance in predicting group and individual-based risky decision-making compared to baselines.
2 Task Formulation and Related Work
Risky decision-making has been studied in many different contexts. Here we formulate  $n$ -choice decision-making problem (nDMP): Taking as input natural language descriptions of  $n$  possible choices/outcomes  $\mathcal{O}$ , choose the most preferred outcome from the set of choices  $\mathcal{O}$ . We focus on a sub-problem known as a  $n$ -choice risky decision-making problem (nRDMP) which is an nDMP where there is some risk or probabilistic outcomes associated with choices in  $\mathcal{O}$ , e.g., programs B and D in the ADP. Specifically, we investigate the gain-loss framing problem which is comprised of two nRDMPs,  $\mathrm{nRDMP}_{\mathrm{gain}}$  where choices are written as gain frames which accentuate the positive features of the text, e.g., programs A and B form a  $2\mathrm{RDMP}_{\mathrm{gain}}$  where 'saving people's lives' is the accentuated feature. Conversely,  $2\mathrm{RDMP}_{\mathrm{loss}}$  where choices are written as loss frames which accentuate the negative features of the text, e.g., programs C and D form a  $2\mathrm{RDMP}_{\mathrm{loss}}$  where 'people dying' is the accentuated feature. Additionally, choices have equivalent outcomes across both 2RDMPs.
2.1 Classical decision theory
Classical decision theory abstracts the outcomes using utilities, which are numerical values that reflect desirability. For example, expected utility theory (EUT) identifies the choice that maximises the expected utility assuming the axioms of rationality (Von Neumann and Morgenstern, 2007). However, in human decision-making these axioms are often violated, giving rise to, e.g., Allais (Allais, 1953) and Ellsberg (Segal, 1987) paradoxes. Generalised EUT such as uncertain utility theory (Gul et al.,
2008), cumulative prospect theory (CPT) (Tversky and Kahneman, 1992), and multiple-criteria decision-making (MCDM) (Zeleny, 2012) were proposed to resolve these discrepancies. However, these classical approaches not only fail to take into account semantic information given by the working of choices which is important contextually for decision making, but they also ignore cognitive processes such as sentiments of decision-makers.
Recent breakthroughs in NLP have led to a revolution in the breadth and robustness of problems that can be solved involving natural language by successfully capturing the underlying semantics and relationships of language. For example, neural language models have found resounding success in representation learning (Mikolov et al., 2013; Devlin et al., 2018), the task of uncovering feature representations of language which are useful for downstream NLP tasks. One such downstream NLP task, sentiment analysis, has benefited largely from the application of language models such as XLNet (Yang et al., 2019) and ULMFiT (Howard and Ruder, 2018). Rapid advancements thus give hope for the development of sophisticated computational decision-making models.
2.2 Group/Individual-level Tasks
In this paper, we consider two specific 2RDMP, group level risky decision making (GL-RDM) which the majority of psychological studies focus on and the novel task of individual level risky decision making (IL-RDM), defined as follows.
GL-RDM: Given a set of observed outcomes from human RDM experiments, each of which is described by a 5-tuple  $(2\mathrm{RDMP}_{gain}, 2\mathrm{RDMP}_{loss}, P_{gain}, P_{loss}, \text{category})$ , where  $2\mathrm{RDMP}_{gain}$  is the gain frame of a 2RDMP,  $P_{gain}$  is the proportion of individuals in the gain frame who chose the risky choice, and category is a grouping of similar experiments based on design and participants described in Section 6.1.  $2\mathrm{RDMP}_{loss}$  and  $P_{loss}$  can be defined similarly by replacing gain with loss. GL-RDM's objective is to predict the distribution of choice between  $P_{gain}$  and  $P_{loss}$  and for unseen human experiments within the same category.
IL-RDM: Given a set of  $n$  RDMPs,  $\mathrm{RDP} = \{rdp_1, rdp_2, \ldots, rdp_n\}$  where gain/loss frames of the same problem can appear as separate RDPs  $rdp_i$ , a set of individuals  $Ivd = ivd_1, ivd_2, \ldots, ivd_m$  and a function which maps individuals and RDPs to their preferred choice
$PC(id_{i}, rdp_{j}) = pc_{i,j}$  where  $pc_{i,j}$  is individual  $id_{i}$ 's preferred choice for  $rdp_{j}$ . The objective for IL-RDM is to learn a model/mapping function for each individual which can predict an individual's preferred choice for unseen nRDMPs.
3 FTT-guided Risky Decision-making
The BR model. Broniatowski and Reyna laid out four main FTT principles in developing a cognitive model, i.e., the  $BR$  model, for the GL-RDM task (Broniatowski and Reyna, 2018, 2014). These principles are: (C1) Decision choices are encoded in different levels of gist representations, e.g., categorical- and interval-levels based on the psychological notion of levels of measurement (Stevens et al., 1946). (C2) Categorical gist representations of choices are distinguished based on binary (positive/negative) sentiments and decision-makers will prefer options with positive associations. In the BR model, sentiments of categories are drawn upon social and moral principles which are stored in long-term memory, e.g., saving lives is fundamentally good. (C3) When comparisons of categorical gist representations do not arrive at a conclusive result, the decision-maker will revert to more precise gist representations. In the BR model, gist representations compete and combine such that the simplest gist representation is chosen. (C4) Categorical gist is encoded based on the decision-maker's prior experiences and individual differences, i.e., need for cognition (NFC), numeracy (NUM), and risk sensitivity (RS).
Human experiments have provided evidence that the BR model is capable of explaining GL-RDM. However, being a box-arrow model, the BR model is comprised of hypothesized concepts or processes that lack precise definitions. Hence applying the model requires human interpretations and judgements on, e.g., notions such as gist lattices of each RDMP, the acquisition of sentiments, and individual differences. This informal nature, along with the inflexibility of the model being unable to be easily adapted to IL-RDM prevents the model from being used as an automated predictive tool.
Our model. Towards a fully automated tool for the RDMP tasks, we propose a computational model of risky decision-making that takes the input text descriptions of an RDMP and solves the GL-RDM and IL-RDM tasks automatically. The model is depicted in Figure 1. The key features of our model include: (1) All model components are automated,
i.e., gist representations are extracted through NLP as categorical embeddings. (2) Categorical and interval representations are formally defined as hierarchical with the interval level encapsulating the properties and information of the categorical-level (Stevens et al., 1946). (3) Individualistic differences, NFC, NUM and RS (see below) directly affect decision-making at a representational level and errors in judgement can propagate through the model adding more expressive.
4 Computing Gist Representations
The first challenge we tackle is the computational encoding of gist representations and how individual differences encode error, a mechanism to model variations in human decision-making at a representation level to perform GL-RDM and IL-RDM.
4.1 Categorical Representations
Categorisation is the act of grouping documents into categories based on semantic or sentiment similarity. For example, the entertainment category in the news dataset (see Section 6.1) comprises of various articles spanning multiple topics such as games, movie reviews, and celebrity gossip. The ability to categorise and recall the underlying sentiments of categories is an important prerequisite for FTT decision making asserted by principle C2 where humans prefer choices associated with categories with positive connotations over negative connotations. For example, the sentiments of the travel category in the news dataset has a strong negative sentiment due to the news articles being collected during the outbreak of COVID-19. Given this negative sentiment, people would be dissuaded from travelling. Current sentiment analysis methods focus on granular extraction of sentiments from text rather than categories where words that are highly indicative of a category do not necessarily reveal any insights into their sentiment.
Vector representations of words (Mikolov et al., 2013), sentences (Devlin et al., 2018), and documents (Le and Mikolov, 2014) capture the semantic relationships between entities. At a higher level, a categorical embedding should capture semantic relations between categories of documents. To our knowledge, no such representation has been proposed. To fill this gap, we propose Category-2-Vector (Cat2Vec) and a sentiment based extension, sentiment-Cat2Vec. Cat2Vec aims to find model-agnostic categorical representations that facilitates
the prediction of categories from text and sentiments from categories. More formally, given a set of  $M$  categories  $C = \{k_1,k_2,\dots ,k_M\}$ , a training set contains a number  $N$  of document-category pairs  $\{(d_{1},c_{1}),\ldots ,(d_{N},c_{N})\}$ , where each  $d_{i}$  is a document and  $c_{i}\in C$  is the (ground truth) category of  $d_{i}$ . The objective is to maximise the average log probability  $\frac{1}{N}\sum_{i = 1}^{N}\log P(c_i|d_i)$  where  $P(c_{i}|d_{i})$  is the probability that document  $d_{i}$  belongs to category  $c_{i}$ . In sentiment-cat2vec,  $P(c_{i}|d_{i})$  is replaced by  $P(s_{i}|c_{i})$ , the probability that category  $c_{i}$  belongs to a certain sentiment class. Here, we consider only binary positive and negative sentiments.
Cat2Vec extends a contrastive learning via negative sampling approach by simultaneously maximising the similarity between document embeddings,  $v_{d_i}$ , with true category embeddings,  $v_{c_i}$ , by minimising the similarity between  $v_{d_i}$  and  $K$  negative category embeddings defined by the objective:
$$
\begin{array}{l} \log \sigma \left(\left(v _ {c _ {i}} \odot v _ {i _ {i}}\right) \cdot v _ {d _ {i}}\right) \\ + \sum_ {j = 1} ^ {K} \mathbb {E} _ {k _ {j} \sim P _ {\text {n o i s e}} (C)} [ \log \sigma ((- v _ {k _ {j}} \odot v _ {i _ {j}}) \cdot v _ {d _ {i}}) ] \tag {1} \\ \end{array}
$$
where  $\odot$  represents element-wise multiplication,  $P_{\mathrm{noise}}(C)$  is a noise distribution that dictates how categories are sampled, we select a uniform distribution,  $\sigma (x) = 1 / (1 + \exp (-x))$  and  $v_{i_i}$  is a category importance vector for category  $i$  which is learned simultaneously with the category embedding which provides an attention-like effect over category by accentuating or diminishing certain features in the category embedding when multiplied together. Furthermore,  $v_{d_i} = Enc(d_i)$  where Enc is a document vector encoding function. In this paper we adopt a bi-directional LSTM with self-attention such that  $v_{d_i} = (\overrightarrow{\alpha}\odot \overrightarrow{v}_{d_i}\| \overleftarrow{\alpha}\odot \overleftarrow{v}_{d_i})$  where  $\vec{\alpha},\vec{\alpha}$  are self-attention weights of the forwards and backwards LSTMs, resp., and  $\vec{v}_{d_i},\vec{v}_{d_i}$  are the hidden state output vectors for document  $d_{i}$  of the forward and backwards LSTMs, respectively. However, the encoder is interchangeable in Cat2Vec e.g. pretrained transformers like BERT (Devlin et al., 2018) can be used.
The novelty of our model lies in two main aspects, the introduction of a category importance vector to improve the ability of the model to learn relations between categories and the ability of our model to estimate  $P(s_{i}|c_{i})$  given labelled text doc
Figure 1: The main architecture of our computational model for decision making based on FTT
uments. To estimate  $P(s_{i}|c_{i})$  we introduce an extra dense output layer (D) in figure 2 which predicts  $P(s|d_{i})$ , the probability that document  $i$  belongs to a certain sentiment class. In the case of binary sentiments, the joint loss becomes the binary cross-entropy loss of predicting the correct sentiment of a document plus the negative sampling loss in equation 1. After training the model we can estimate  $P(s_{i}|c_{i})$  by feeding the learned category embeddings,  $(v_{c_i} \odot v_{i_i})$  into the output layers (D). Although these output layers are trained to learn  $P(s|d_{i})$ , since the learned category embeddings are based on the document embeddings and are learned in the same semantic space, this approach gives us good estimates of  $P(s_{i}|c_{i})$ .
Figure 2: Cat2Vec Model
Equation 2 shows how error encoded categorical utility (CU) is calculated from categorical representations where Category(choice) is a function that takes RDM choices as inputs and outputs the underlying category related to the choice, Sentiment(category) is a function which takes a category as input and outputs the underlying sentiments related to that category as  $\text{pos}_{\text{category}} - \text{neg}_{\text{category}}$ . Categorical error is encoded based on NFC, an individual's tendency to engage in and enjoy cognitive activities (Cacioppo et al., 1996), can introduce error at a categorical level to account for individualism. To calculate an error encoded CU, we
sample from a logistic distribution which is consistent with existing literature in qualitative discrete choice models (McFadden, 2001). Formally,  $\mathrm{NFC} \in (0,1)$  and  $CU \sim \operatorname{Logistic}(\mu, s)$  where  $\mu = \mathbb{E}[X]$  is the expected or true utility value and  $s(\mathrm{NFC}) = |\mathrm{NFC} - 1| \times \mathbb{E}[X]$ .
$$
\mathrm {C U} = \text {L o g i s t i c} (\text {S e n t i m e n t} (\text {C a t e g o r y} (\text {c h o i c e})) \text {, N F C}) \tag {2}
$$
4.2 Interval Representations
Interval representations are a more precise representation than categorical representation. It encodes the calculation of the expected value (EV) and utility of choices. Numerical information from text can be extracted using simple text extraction or named entity recognition (NER) (Nadeau and Sekine, 2007) where probabilities and their associated quantities can be extracted as arrays, e.g., in program B of the RDMP in the introduction the probabilities would be  $[1/3, 2/3]$  and their corresponding quantities would be  $[600, 0]$ .
Equation 3 outlines the process to generate error-encoded interval utilities where CU is the categorical utility defined in equation 2 and EV is an expected value function which takes an input RDM choices and outputs the corresponding expected value associated with probabilities and quantities in choices which can extracted using standard text identification techniques such named entity recognition. Error is encoded based on NUM (Kahneman, 2003), which measures a person's ability to interpret and work with numbers to account for individualism is calculated as  $IU \sim \operatorname{Logistic}(\mu, s)$  where  $\mathrm{NUM} \in (0,1)$ ,  $\mu = \mathbb{E}[X]$  is the true expected value and  $s(\mathrm{NUM}) = Q|\mathrm{NUM} - 1| \times \mathbb{E}[X]$  where  $Q$  is number of quantities in the choice to account for error involving multiple calculations.
$$
\mathrm {I U} = \text {L o g i s i t i c} (\mathrm {E V} (\text {c h o i c e}), \mathrm {N U M}) \cdot \mathrm {C U} \tag {3}
$$
4.3 Representations for Decision Making
Finally, combining these representations allows us to derive the most beneficial choice in an nRDMP. The preferred categorical,  $\mathrm{Pref}_{\mathrm{Cat}}$  and preferred interval,  $\mathrm{Pref}_{\mathrm{Int}}$  choices are calculated based on which choice maximises categorical and interval utilities, respectively. If  $\mathrm{Pref}_{\mathrm{Cat}} = \mathrm{Pref}_{\mathrm{Int}}$ , there is a consensus on the best choice. If  $\mathrm{Pref}_{\mathrm{Cat}} \neq \mathrm{Pref}_{\mathrm{Int}}$ , there is no clear best choice. In this case,  $RS$ , a person's preference towards pursuing riskier but more rewarding decisions (Kacelnik and Bateson, 1997), is adopted as in the BR model. Risk sensitivity influences the probability of choosing the safest or riskiest choice in an nRDMP as  $P(\text{risky}) = 1 / (1 + e^{-RS})$  where  $RS \in (-3,3)$ . The safest choice is one that involves the least probabilistic outcomes, whereas conversely, the riskiest choice involves the most probabilistic outcomes e.g., in the ADP in the introduction, program A is the safest as it involves one certain outcome while program B is the riskiest with two probabilistic outcomes.
4.4 Decision Making: A Worked Example
To demonstrate the fluidity of our model we apply our model to the ADP from the introduction. In the gain frame, the predicted category of programs A and B using the pretrained Cat2Vec from the experiments predicts the life category for both programs. The sentiments of the life category predicted by Sentiment-Cat2Vec is 0.9999 positive and 0.0001 negative giving categorical utility defined as  $\text{pos}_{\text{category}} - \text{neg}_{\text{category}}$  for both programs equal to 0.9998. Taking into account numerical information, the expected value of programs A and B is 200 people being saved, the interval utility is thus the expected value times the categorical utility which is 199.6 for both programs. No consensus between categorical or interval choices can be made due to unclear preferred choices for both. Thus, the final choice is decided by risk sensitivity.