Affective Natural Language Generation of Event Descriptions through Fine-grained Appraisal Conditions (2023)

Abstract: Models for affective text generation have shown a remarkable progress, but they commonly rely only on basic emotion theories or valance/arousal values as conditions. This is appropriate when the goal is to create explicit emotion statements (“The kid is happy.”). Emotions are, however, commonly communicated implicitly. For instance, the emotional interpretation of an event (“Their dog died.”) does often not require an explicit emotion statement. In psychology, appraisal theories explain the link between a cognitive evaluation of an event and the potentially developed emotion. They put the assessment of the situation on the spot, for instance regarding the own control or the responsibility for what happens. We hypothesize and subsequently show that including appraisal variables as conditions in a generation framework comes with two advantages. (1) The generation model is informed in greater detail about what makes a specific emotion and what properties it has. This leads to text generation that better fulfills the condition. (2) The variables of appraisal allow a user to perform a more fine-grained control of the generated text, by stating properties of a situation instead of only providing the emotion category. Our Bart and T5-based experiments with 7 emotions (Anger, Disgust, Fear, Guilt, Joy, Sadness, Shame), and 7 appraisals (Attention, Responsibility, Control, Circumstance, Pleasantness, Effort, Certainty) show that (1) adding appraisals during training improves the accurateness of the generated texts by 10 pp in F1. Further, (2) the texts with appraisal variables are longer and contain more details. This exemplifies the greater control for users.

## introduction the main task of conditional natural language generation (cnlg) is to provide freedom to control the output text. it is commonly addressed as the intersection of text-to-text @xcite condition: joy responsibility output: i won the tournament due to extensive training. figure 1 : conditioning text generation on emotions (blue) and appraisals (green) results in an improved fulfillment of the emotion condition by incorporating event descriptions (green) in the output text. this enables more fine-grained control over the generated text. @xcite and datato-text generation @xcite @xcite . therefore, models typically use two inputs: a textual triggerphrase, and a condition to guide the generation. in affective cnlg models, the condition is an affective state, typically represented as valence/arousal values @xcite or discrete emotion names @xcite . arguably, the use of theories of basic emotions @xcite is appropriate when the main requirement is to express a particular emotion. however, a natural communication of emotions also includes implicit expressions, where the main content of a message is not (only) the emotion. as an example, humans describe an event and leave it to the dialogue partner to infer the affective meaning ("yesterday, my dog died"). in fact, @xcite report that event descriptions are used to convey an emotion in 75 % of instances in the tec corpus @xcite : the sentence "i won money in the lottery" does, for most people, not require a mention of the associated emotion. in this paper, we focus on the task of generating such emotionally connotated event descriptions (figure 1 ). this poses the challenge how to represent the link between "factual" events and their emotion. appraisal theories from psychology attempt to explain that connection with variables that represent the cognitive evaluation by a person in context of a situation @xcite . does the person feel respon-sible? do they pay attention to what is going on? is the event pleasant? does somebody have control over what is happening? how much effort is needed to deal with the outcome of the situation? these variables explain emotions: feeling responsible is a prerequisite for feeling guilty, not knowing about the outcome of a potentially negative event might cause fear (while knowing about it is more likely to cause sadness). our paper has two main contributions: (1) we hypothesize and show that providing appraisal information along the emotion category to the model, leads to a better fulfillment of the emotion condition. @xcite we show that adding appraisal variables leads to a more fine-grained control of the generation process and the resulting texts show more details regarding the described event. 1 2 related work ## methods the objective of our paper is to understand if adding appraisal information in addition to emotion conditions to a generator (1) improves the accuracy of the output, i.e., the likelihood that the output in fact exhibits the target emotion. further, (2), we aim at understanding if these appraisal variables provide a more fine-grained control to the users (e.g., "i am relaxed" vs. "i am relaxed because i worked for only 6 hours"). to address these goals, we configure three cnlg models ( training. in each configuration, we embed the conditions in the input prompt, to fine-tune the models. this strategy avoids expensive trainingencoders or decoders, or both-with condition information from scratch. we create training data out of existing corpora that are labeled for emotions and appraisals consisting of input prompts and output pairs. the input prompt contains the conditions (e.g., joy; joy attention), as special tokens, followed by the trigger-phrase (e.g., last day i). the output are the remaining words the model should learn to produce (e.g., was relaxed because i worked for 6 hours). this leads to the following three prompt representations (see by using non-special tokens to represent the target conditions, the models can make use of knowledge acquired in pretraining. we opt for a string representation over a numerical representation (e.g., "control" instead of "1" or "nocont" instead of "0"), because preliminary experiments showed that numerical representations are sometimes interpreted as a request for repetitions by t5 ("generate 1 1: i feel" → "i feel i feel"). inference. at prediction time, we obtain the five most probable sentences for each prompt. these sentences are selected using beam search @xcite with beam size 30, next token temperature of 0.7, top-p foot_1 (nucleus) sample of 0.7. we ensure that our output excludes sentences with repeated instances of the same bigram. ## experiments the following subsections explain the experiments conducted to test our hypotheses. in §4.1, we describe the setting and fine-tuning of the models. in §4.2, we provide results to answer the question (1) if appraisals in conjunction with emotion conditions improve the generation such that it meets the emotion condition. in §4.3 we discuss results to understand if appraisals are a means for a more fine-grained control of the generation process. ## human evaluation we conduct a human study to validate the automatic evaluation. further, this study assesses additional measures, namely the quality of the generated text. we focus on the best-performing model, t5, finetuned in the ea and e configurations. setup. we randomly select 100 sentences from the following model-configuration and testing prompt set combinations: ea with ep, e with ep, and ea with efa. in addition, we include 30 sentences from the crowd-envent dataset to confirm the validity of the crowd-working setup. these 30 sentences are selected to be "easily-annotated" based on a high inter-annotator agreement in the original data. we evaluate the 330 sentences on the platform https://www.soscisurvey.de . the survey consists of 23 statements to be rated on a five-level likert scale. seven statements correspond to the emotions ("what do you think the writer of the text felt when experiencing this event?"). seven statements correspond to the appraisal variables ("how much do these statements apply?"), and seven questions measure the text quality (fluency, grammaticality, being written by a native speaker, semantical coherence, realistic event, written by an artificial intelligence, written by a human). in addition, we include two attention checks. we recruit participants via https://www.prolific.co/ . §c.1 shows the questions in detail. results. to compare the performance of the conditional natural language generation models, using the human evaluation (five-level), we discretize emotion and appraisal scores, analogously to the discretization of the crowd-envent labels for our conditional models. we assign the labels based on a majority vote of three annotators. the two rows for the ep testing prompt (with ea and e model configurations) also mimic the automatic evaluation. this is, however, not shown in the average f 1 score because the differences are less pronounced. nevertheless, we observe that all emotions are better generated with the ea model than with the e model, except for guilt and shame. therefore, the human evaluation confirms that training models with appraisal information lead to a better generation of emotion-bearing sentences. we report results for appraisals in appendix c.2. importantly, the text mostly remains coherent. ## conclusion and future work we presented the first study on conditional text generation based on both basic emotion category names and appraisal theories. we find that the emotion is more reliably represented when appraisals are provided during training, even when the appraisals are not provided during inference. in addition, we provide evidence that the combination of appraisals enables a more fine-grained control over the generated text. by switching the appraisal variables, distinct event descriptions are produced, even when the emotion remains constant. this leads to important future work: while we believe that appraisals shall be used to generate more detailed and accurate texts, the decrease in text quality needs to be controlled. in our work, we relied on prompt-based representations of the conditions in the generator models. different model architectures (e.g., embedding the condition into the encoder, decoder, or both) could improve or maintain the quality of the generated text. in our experiments, we relied on annotated data with labels that we used as conditions. in these data, all variables were always accessible. in a real-world setup, a deployable model would need to automatically estimate (a subset of) appraisal dimensions or request required information from a user. this might lead to a novel setup of conditioning under partial information which poses new challenges for general models of conditional text generation. finally, we left the topic of the event description to the choice of the model. in a real-world setup, additional conditions need to be included, for instance a topic, or a previous utterance in a dialogue. these various conditions might be in conflict in the context of a dialogue, and the model would need to rank (automatically) the conditions.