SlowGuess/NAACL_2021_OCR huggingface dataset:
I-Ta Lee, Maria Leonor Pacheco, Dan Goldwasser
West Lafayette, IN, USA
{lee2226, pachecog, dgoldwas} @ purdue.edu
Understanding narrative text requires capturing characters' motivations, goals, and mental states. This paper proposes an Entity-based Narrative Graph (ENG) to model the internal-states of characters in a story. We explicitly model entities, their interactions and the context in which they appear, and learn rich representations for them. We experiment with different task-adaptive pre-training objectives, in-domain training, and symbolic inference to capture dependencies between different decisions in the output space. We evaluate our model on two narrative understanding tasks: predicting character mental states, and desire fulfillment, and conduct a qualitative analysis.
Understanding narrative text requires modeling the motivations, goals and internal states of the characters described in it. These elements can help explain intentional behavior and capture causal connections between the characters' actions and their goals. While this is straightforward for humans, machine readers often struggle as a correct analysis relies on making long range common-sense inferences over the narrative text. Providing the appropriate narrative representation for making such inferences is therefore a key component. In this paper, we suggest a novel narrative representation model and evaluate it on two narrative understanding tasks, analyzing the characters' mental states and motivations (Abdul-Mageed and Ungar, 2017; Rashkin et al., 2018; Chen et al., 2020), and desire fulfillment (Chaturvedi et al., 2016; Rahimtoroghi et al., 2017).
We follow the observation that narrative understanding requires an expressive representation capturing the context in which events appear and the interactions between characters' states. To clarify, consider the short story in Fig. 1. The desire expression appears early in the story and provides
the context explaining the protagonist's actions. Evaluating the fulfilment status of this expression,
Figure 1: Narrative Example
which tends to appear towards the end of the story, requires models that can reason over the desire expression ("trying something new"), its target ("apples") and the outcome of the protagonist's actions ("it's now her favorite apple dish!). Capturing the interaction between the motivation underlying the desire expression (in Fig. 1, CURIOSITY) and the emotions (in Fig. 1, ANTICIPATION) likely to be invoked by the motivation can help ensure the consistency of this analysis and improve its quality.
To meet this challenge, we suggest a graph-contextualized representation for entity states. Similar to contextualized word representations (Peters et al., 2018; Devlin et al., 2019), we suggest learning an entity-based representation which captures the narrative it is a part of. For example, in "She decided to try to make baked apples for the first time" the mental state of "she" would be represented differently given a different context, such as a different motivation for the action ("Her mother asked her to make an apple dish for a dinner party"). In this case, the contextualized representation would capture the different emotion associated with it (e.g., FEAR of disappointing her mother). Unlike contextualized word embeddings, entity-based contextualization needs to consider, at least, two levels of context: local text context and distant event context, which require more complicated modeling techniques to capture event semantics. Moreover,
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4916-4926 June 6-11, 2021. ©2021 Association for Computational Linguistics
the context of event relationships can spread over a long narrative, exceeding maximum sequence length limitation in modern contextualized word embedding models such as BERT (Devlin et al., 2019).
In this paper, we propose an Entity-based Narrative Graph (ENG) representation of the text. Unlike other graph-based narrative representations (Lehnert, 1981; Goyal et al., 2010; Elson, 2012) which require intensive human annotation, we design our models around low-cost supervision sources and shift the focus from symbolic graph representations of nuanced information to their learned embedding. In ENG, each node is associated with an entity-event pair, representing an entity mention that is involved in an event. Edges represent observed relations between entities or events. We adapt the definition of event relationships introduced in Lee et al. (2020) to our entity-event scenario. For entity relationships, the CNext relationship connects two coreferent entity nodes. For event relationships, the Next relationship captures the sequential order of events as they appear in the text, and six discourse relation types from the Penn Discourse Tree Bank (PDTB) (Prasad et al., 2007) are used. These include Before, After, Sync., Contrast, Reason and Result. Note that these are extracted in a weakly supervised manner, without expensive human annotations.
To contextualize the entity embeddings over ENG, we apply a Relational Graph Convolution Network (R-GCN) (Schlichtkrull et al., 2018), a relational variant of the Graph Convolution Network architecture (GCN) (Kipf and Welling, 2016). R-GCNs create contextualized node representations by considering the graph structure through graph convolutions and learn a composition function. This architecture allows us to take into account the narrative structure and the different discourse relations connecting the entity-event nodes.
To further enhance our model, we investigate three possible pre-training paradigms: whole-wordmasking, node prediction, and link prediction. All of them are constructed by automatically extracting noisy supervision and pre-training on a largescale corpus. We show that choosing the right pre-training strategy can lead to significant performance enhancements in downstream tasks. For example, automatically extracting sentiment for entities can impact downstream emotion predictions. Finally, we explore the use of a symbolic inference
layer to model relationships in the output space, and show that we can obtain additional gains in the downstream tasks that have strong correlation in the output space.
The evaluated downstream tasks include two challenging narrative analysis tasks, predicting characters' psychological states (Rashkin et al., 2018) and desire fulfilment (Rahimtoroghi et al., 2017). Results show that our model can outperform competitive transformer-based representations of the narrative text, suggesting that explicitly modeling the relational structure of entities and events is beneficial. Our code and trained models are publicly available<sup>1</sup>.
Tracking entities and modeling their properties has proven successful in a wide range of tasks, including language modeling (Ji et al., 2017), question answering (Henaff et al., 2017) and text generation (Bosselut et al., 2018). In an effort to model complex story dynamics in text, Rashkin et al. (2018) released a dataset for tracking the emotional reactions of characters in stories. In their dataset, each character mention is annotated with three types of mental state descriptors: Maslow's "hierarchy of needs" (Maslow, 1943), Reiss' "basic motives" (Reiss, 2004), that provide a more informative range of motivations, and Plutchik's "wheel of emotions" (Plutchik, 1980), comprised of eight basic emotional dimensions (e.g. joy, sadness, etc). In their paper, they showed that neural models with explicit or latent entity representations achieve promising results on this task. Paul and Frank (2019) approached this task by extracting multi-hop relational paths from ConceptNet, while Gaonkar et al. (2020) leveraged semantics of the emotional states by embedding their textual description and modeling the co-relation between different entity states. Rahimtoroghi et al. (2017) introduced a dataset for the task of desire fulfillment. They identified desire expressions in first-person narratives and annotated their fulfillment status. They showed that models that capture the flow of the narrative perform well on this task.
Representing the narrative flow of stories using graph structures and multi-relational embeddings has been studied in the context of script learning (Li et al., 2018; Lee and Goldwasser, 2019; Lee et al.,
1https://github.com/doug919-entity-based_narrative_graph
2020). In these cases, the nodes represent predicate-centric events, and entity mentions are added as context to the events. In this paper, we use an entity-centric narrative graph, where nodes are defined by entity mentions and their textual context. We encode the textual information in the nodes using pre-trained language models (Devlin et al., 2019; Liu et al., 2019), and the graph structure with a relational graph neural network (Schlichtkrull et al., 2018). To learn the representation, we incorporate a task-adaptive pre-training phase. Gururangan et al. (2020) showed that further specializing large pre-trained language models to domains and tasks within those domains is effective.
3 Entity-based Narrative Graph
Many NLU applications require understanding entity states in order to make sophisticated inferences (Sap et al., 2018; Bosselut et al., 2019; Rashkin et al., 2018), and the entity states are highly related to the event the entity involves in. In this work, we propose a learning framework that aims at modeling entities' internal states, and their interactions to other entities' internal states through events. We include task-adaptive pretraining (TAPT) and downstream task training to train an entity-based narrative graph (ENG), a graph neural model designed to capture implicit states and interactions between entities. We extend the narrative graph proposed by Lee et al. (2020), which models event relationships, and instead of learning node representations for events, we focus on entity mentions that are involved in events. This change is motivated by the high-demand of NLU applications that require understanding entity mentions' states in order to make sophisticated inference.
Our framework consists of four main components: Node Encoder, Graph Encoder, Learning Objectives, and Symbolic Inference, outlined in Figure 2. The node encoder is a function used to extract event information about the target entity mention corresponding to the local node representation. The graph encoder uses a graph neural network to contextualize node representations with entity-events in the same document, generating entity-context-aware representations. The learning objectives use this representation for several learning tasks, such as node classification, link prediction, and document classification. Finally, we
include a symbolic inference procedure to capture dependencies between output decisions.
We introduce a training pipeline, containing pretraining and downstream training, following recent evidence suggesting that task-adaptive pre-training is potentially useful for many NLU tasks (Gururangan et al., 2020). We experiment with three pre-training setups, including the common whole-word-masking pre-training (Liu et al., 2019), and two newly proposed unsupervised pre-training objectives based on ENG. We then evaluate two downstream tasks: StoryCommonsense (Rashkin et al., 2018) and DesireDB (Rahimtoroghi et al., 2017). StoryCommonsense aims at predicting three sets of mental states based on psychological theories (Maslow, 1943; Reiss, 2004; Plutchik, 1980), while DesireDB's goal is to identify whether a target desire is satisfied or not. Solving these tasks requires understanding entities' mental states and their interactions.
Figure 2: Overview of the ENG framework.
3.2 Node Encoder
Each node in our graph captures the local context of a specific entity mention (or character mention), and how the entity mentions are extracted is related to extracting their edges, which will be described in Sec. 3.3. Following Gaonkar et al. (2020), we format the input information to be fed into a pretrained language model. For a given character  $c$  and sentence  $s$ , the inputs to the node encoder consist of three components  $(s, ctx(c), L)$ , where  $s$  is the sentence in which  $c$  appears,  $ctx(c)$  is the context of  $c$  (all the sentences that the character appears in), and  $L$  is a label sentence. The label sentence is an artificial sentence of the form "[entity name] is [label 1], [label 2], ..., [label k]." The  $k$  labels correspond to the target labels in the downstream task. For example, in StoryCommonsense, the Plutchik state prediction task has eight labels characteriz-
ing human emotions, such as joy, trust, and anger. Gaonkar et al. (2020) show that self-attention is an effective way to let the model take label semantics into account, and improve performance $^2$ .
Our best model uses RoBERTa (Liu et al., 2019), a highly-optimized version of BERT (Devlin et al., 2019), to encode nodes. We convert the node input  $(s, ctx(c), L)$  to RoBERTa's two-sentence input format by treating  $s$  as the first sentence, and the concatenation of  $ctx(c)$  and  $L$  as the second sentence. After forward propagation, we take the pooled sentence representation (i.e.,  $< s>$  for RoBERTa, CLS for BERT), as the node representation  $v$ . This is formulated as  $v = f_{roberta}(s, ctx(c), L)$ .
The ENG is defined as  $ENG = (V, E)$ , where  $V$  is the set of encoded nodes in a document and  $E$  is the set of edges capturing relationships between nodes. Each edge  $e \in E$  is a triplet  $(v_1, r, v_2)$ , where  $v_1, v_2 \in V$  and  $r$  is an edge type  $(r \in R)$ . Following Lee et al. (2020), we use eight relation types  $(|R| = 8)$  that have been shown to be useful for modeling narratives. NEXT denotes if two nodes appear in neighboring sentences. CNEXT expresses the next occurrence of a specific entity following its co-reference chain. Six discourse relation types, used by Lee et al. (2020) and defined in Penn Discourse Tree Bank (PDTB) (Prasad et al., 2007), are also used in this work, including BEFORE, AFTER, SYNC., CONTRAST, REASON, RESULT. Their corresponding definition in PDTB and can be found in Table 1. Following Lee et al. (2020), we use the Stanford CoreNLP pipeline<sup>3</sup> (Manning et al., 2014) to obtain co-reference links and dependency trees. We use them as heuristics to extract the above relations and identify entities for TAPT<sup>4</sup>. Details of this procedure can be found in (Lee et al., 2020). Note that although we share the same relation definitions, our nodes are defined over entities, instead of events.
For encoding the graph, we use a Relational Graph Convolution Network (R-GCN) (Schlichtkrull et al., 2018), which is designed for Knowledge Base Completion. This
Abbrev.	PDTB	Distr.
NEXT	-	50%
CNEXT	-	20%
BEFORE	Temporal.Async.Precedence	5%
AFTER	Temporal.Async.Succession	5%
SYNC.	Temporal.Synchrony	5%
CONTRAST	Comparison.Contrast	5%
REASON	Contingency.Cause.Reason	5%
RESULT	Contingency.Cause.Result	5%
Table 1: Alignment between PDTB relations and the abbreviations used in this paper. The third column in the sampling distribution.
architecture is capable of modeling typed edges and is resilient to noise. R-GCN is defined as:
$$
h _ {i} ^ {l + 1} = R e L U \left(\sum_ {r \in R} \sum_ {u \in U _ {r} (v _ {i})} \frac {1}{z _ {i , r}} W _ {r} ^ {l} h _ {u} ^ {l}\right), \tag {1}
$$
where  $h_i^l$  is the hidden representation for the i-th node at layer  $l$  and  $h_i^0 = v_i$  (output of the node encoder);  $U_r(v_i)$  represents  $v_i$ 's neighboring nodes connected by the relation type  $r$ ;  $z_{i,r}$  is for normalization; and  $W_r^l$  represents trainable parameters.
Our implementation of R-GCN propagates messages between entity nodes, emulating the interactions between their psychological states, and thus enriching node representations with context. Note that our framework is flexible, and alternative node and graph encoders could be used.
3.4 Output Layers and Learning Objectives
We explore three learning problem types.
Node Classification For node classification, we use the contextualized node embeddings coming from the graph encoder, and plug in a  $k$ -layer feedforward neural network on top ( $k = 2$  in our case). The learning objectives could be either multi-class or multi-label. For multi-class classification, we use the weighted cross-entropy loss (CE). For multi-label classification, we use the binary cross-entropy (BCE) loss for each label:
$$
C E = - \frac {1}{N} \sum_ {i = 1} ^ {N} \alpha_ {i} y _ {i} \log (S (g (f (x _ {i}))) \tag {2}
$$
where  $S(.)$  is the Softmax function,  $f(.)$  is the graph encoder,  $g(.)$  is the node encoder,  $x_{i}$  is the input including the target node  $i$  ((s, ctx(c), L)) and all other nodes in the same document (or ENG),  $y_{i}$  is the label, and  $\alpha_{i}$  is the example weight based on the label distribution of the training set..
<sup>5</sup>We tried weighted an unweighted BCE, and selected the unweighted one for our final model.
2Note that all candidate labels are appended to every example, without denoting which one is the right answer. Our preliminary experiments confirm that taking label semantics into account improves performance
3Stanford CoreNLP v4.0 with default annotators.
4For StoryCommonsense, since the entity names are annotated, we simply use them.
Link Prediction This objective tries to recover missing links in a given ENG. We sample a small portion of edges (20% in our case) as positive examples, based on the relation type distribution given in Table 1, taken from the training set. To obtain negative examples, we corrupt the positive examples by replacing one component of the edge triplet with a sampled component so that the resulting triplet does not exist in the original graph. For example, given a positive edge  $(e_1, r, e_2)$ , we can create negative edges:  $(e_1', r, e_2)$ ,  $(e_1, r', e_2)$ , or  $(e_1, r, e_2')$ . Following Schlichtkrull et al. (2018), we score each edge sample with DistMult (Chang et al., 2014):
$$
D (i, r, j) = h _ {i} ^ {T} W _ {r} h _ {j}, \tag {3}
$$
where  $W_{r}$  is a relation-specific trainable matrix (non-diagonal) and  $h_{i}$  and  $h_{j}$  are node embeddings coming from the graph encoder. A higher score indicates that the edge is more likely to be active. To learn this, we reward positive samples and penalize negative ones, using an adapted CE loss:
$$
\begin{array}{l} L = -\frac{1}{|T|}\sum_{(i,r,j,y)\in T}y\log (\sigma (\epsilon_ {r}D(i,r,j))) \\ + (1 - y) \log (1 - \sigma \left(\epsilon_ {r} D (i, r, j)\right)), \tag {4} \\ \end{array}
$$
$T$  is the sampled edges set,  $y = \{0,1\}$ ,  $\sigma(.)$  is the Sigmoid function, and  $\epsilon_r$  is the edge type weight, based on the edge sampling rate in Table 1.
Document Classification For document classifications, such as DesireDB, we aggregate the node representations from the entire ENG to form a single representation. To leverage the relative importance of each node, we add a self-attention layer on top of the graph nodes. We calculate the attention weights by attending on the query embedding (in DesireDB, this is the sentence embedding for the desire expression).
$$
\begin{array}{l} a _ {i} = R e L U \left(W _ {a} \left[ h _ {i}; h _ {t} \right] + b _ {a}\right) \\ z _ {i} = \exp (a _ {i}) \\ \alpha_ {i} = \frac {z _ {i}}{\sum_ {k} z _ {k}}; \quad h _ {d} = \sum_ {i} \alpha_ {i} h _ {i} \tag {5} \\ \end{array}
$$
where  $h_i$  is the i-th node representation,  $h_t$  is the query embedding,  $W_a$  and  $b_a$  are trainable parameters, and  $h_d$  is the final document representation. We then feed  $h_d$  to a two-hidden-layer classifier to make predictions. We use the loss function specified in Eq. 2.
3.5 Task-Adaptive Pre-training
Recent studies demonstrate that downstream tasks performance can be improved by performing self-supervised pre-training on the text of the target domain (Gururangan et al., 2020), called Task-Adaptive Pre-Training (TAPT). To investigate whether different TAPT objectives can provide different insights for downstream tasks, we apply three possible pre-training paradigms and compare them on StoryCommonsense. We focus on StoryCommonsense given that the dataset was created by annotating characters' mental states on a subset of RocStories (Mostafazadeh et al., 2016), a corpus with 90K short common-sense stories. This provides us with a large unlabeled resource for investigating different pre-training methods. We run TAPT on all the RocStories text<sup>6</sup>. We use the learning parameters suggested by Gururangan et al. (2020) and explore the following strategies:
Whole-Word Masking: Randomly masks a subset of words and asks the model to recover them from their context (Radford et al., 2019; Liu et al., 2019). We perform this task over RoBERTa, initialized with roberta-base.
ENG Link Prediction: Weakly-supervised TAPT over the ENG. The setup follows Sec. 3.4 (Link Prediction) to learn a model that can recover missing edges in the ENG.
ENG Node Sentiment Classification: Performs weakly-supervised sentiment TAPT. We use the Vader sentiment analysis (Hutto and Gilbert, 2014) tool to annotate the sentiment polarity for each node in the ENG, based on its sentence. The setup follows Sec. 3.4 (Node Classification).
3.6 Symbolic Inference
In addition to modeling the narrative structure in the embedding space, we add a symbolic inference procedure to capture structural dependencies in the output space for the StoryCommonsense task. To model these dependencies, we use DRaiL (Pacheco and Goldwasser, 2021), a neural-symbolic framework that allows us to define probabilistic logical rules on top of neural network potentials.
Decisions in DRaiL are modeled using rules, which can be weighted (i.e., soft constraints), or unweighted (i.e., hard constraints). Rules are formatted as horn clauses:  $\mathrm{A} \Rightarrow \mathrm{B}$ , where A is a conjunction of observations and predicted values, and
$^{6}$ Not including the validation and testing sets of Story Cloze Test
B is the output to be predicted. Each weighted rule is associated with a neural architecture, which is used as a scoring function to obtain the rule weight. The collection of rules represents the global decision, and the solution is obtained by performing MAP inference. Given that rules are written as horn clauses, they can be expressed as linear inequalities corresponding to their disjunctive form, and thus MAP inference is defined as a linear program.
In DRaiL, parameters are trained using the structured hinge loss. This way, all neural parameters are updated to optimize the global objective. Additional details can be found in (Pacheco and Goldwasser, 2021). To score weighted rules, we used feed-forward networks over the node embeddings obtained by the objectives outlined in Sec. 3.4 and 3.5, without back-propagating to the full graph. We model the following rules:
Weighted rules We score each state, as well as state transitions to capture the progression in a character's mental state throughout the story.
$$
\begin{array}{l} \operatorname {E n t i t y} \left(\mathbf {e} _ {\mathrm {i}}\right) \Rightarrow \operatorname {S t a t e} \left(\mathbf {e} _ {\mathrm {i}}, \mathbf {l} _ {\mathrm {i}}\right) \\ \operatorname {S t a t e} \left(\mathbf {e} _ {\mathrm {i}}, \mathbf {l} _ {\mathrm {i}}\right) \wedge \operatorname {H a s N e x t} \left(\mathbf {e} _ {\mathrm {i}}, \mathbf {e} _ {\mathrm {j}}\right) \Rightarrow \operatorname {S t a t e} \left(\mathbf {e} _ {\mathrm {j}}, \mathbf {l} _ {\mathrm {j}}\right) \\ \end{array}
$$
where  $e_i$  and  $e_j$  are two different mentions of the same character, and HasNext is a relation between consecutive sentences. State can be either Maslow, Reiss or Plutchik.