Eliciting Affective Events from Language Models by Multiple View Co-prompting (2023)

Abstract: Prior research on affective event classification showed that exploiting weakly labeled data for training can improve model performance. In this work, we propose a simpler and more effective approach for generating training data by automatically acquiring and labeling affective events with Multiple View Co-prompting, which leverages two language model prompts that provide independent views of an event. The approach starts with a modest amount of gold data and prompts pre-trained language models to generate new events. Next, information about the probable affective polarity of each event is collected from two complementary language model prompts and jointly used to assign polarity labels. Experimental results on two datasets show that the newly acquired events improve a state-of-the-art affective event classifier. We also present analyses which show that using multiple views produces polarity labels of higher quality than either view on its own.

## introduction people's emotional states are influenced by the events that they experience. for example, people typically feel happy when they graduate with a degree or get a new job, but become upset when they get fired or lose personal property. prior work @xcite has referred to events that positively or negatively impact people as affective events. in this work, we study the task of affective event classification, which determines whether the polarity of a given event is positive, negative or neutral. for example, "i graduated from college" would be positive, but "i broke my leg" would be negative. previous research has shown that the performance of affective event classification models is limited by the amount of gold training data @xcite , which is costly to annotate and not readily available in large quantities. recently, re-searchers have been developing methods to generate more training data by extracting events from text corpora and assigning polarity labels with weakly supervised methods @xcite . however, these methods pose practical challenges, including in some cases the need to acquire data from twitter @xcite , a computational bottleneck of applying a pipeline of nlp tools to a large text collection, and the limitations of lexical pattern matching. in this work, we propose a simpler but more effective approach for automatically acquiring affective events by prompting pre-trained language models. we use one language model prompt to elicit affective event candidates, and we introduce a co-prompting method to automatically label these event candidates with affective polarity. the key idea behind co-prompting is to design two complementary prompts that capture independent views of an event, reminiscent of co-training @xcite . combining information from two different views of an event produces labels that are more accurate than the labels assigned by either one alone. specifically, we acquire affective events in a twostep process: (1) event generation and (2) polarity labeling. the first step generates events that are associated with a set of gold "seed" affective events. for each seed event, we prompt a language model to generate sentences where the seed event co-occurs with some new events. our hypothesis is that affective events are often preceded or followed by other affective events that are causally or temporally related. for example, if someone breaks his/her leg, a prior event might describe how it happened (e.g., "fell off a ladder" or "hit by a car") and a subsequent event might describe the consequences (e.g., "could not walk" or "rushed to the hospital"). the second step collects independent views of the polarity for each new event using two comple-mentary language model prompts. one prompt provides an associated event view, which considers the polarities of the known (labeled) events that co-occur with the new event during event generation. the second prompt provides an emotion view, which considers the polarity of the most probable emotion words generated by a language model when prompted with the new event. finally, we combine information from the two co-prompts to assign an affective polarity label to each new event. our experiments show that using these automatically acquired affective events as additional training data for an affective event classifier produces state-of-the-art performance over two benchmark datasets for this task. the analysis also confirms that our co-prompting method utilizing multiple views yields more accurate polarity labels than using either view alone. in summary, the contributions of our work are: ## related work several lines of research have recognized the importance of identifying events that carry affective polarity, including early work on plot units @xcite and later work that learned patient polarity verbs @xcite , emotionprovoking events @xcite , patterns associated with first-person affect @xcite , major life events @xcite and +/-effects for opinion analysis @xcite @xcite . recent work has focused specifically on classifying affective event phrases @xcite @xcite . ding and riloff (2016) created a weakly supervised method for labeling events with affective polarity using label propagation. ding and riloff (2018) subsequently developed a method that assigns affective polarity to events by optimizing for semantic consistency over a graph structure, and created an affective event knowledge base (aekb) of more than half a million event phrases labeled with affective polarity. zhuang et al. (2020) later created an aff-bert classifier that substantially outperformed aekb for affective event classification by training bert with a relatively small amount of gold data. they additionally developed a discourse-enhanced self-training (dest) method that further improved aff-bert's performance. their approach used twitter to collect events that corefer with sentiment expressions in specific lexical patterns. our work also aims to improve a classification model with automatically generated affective events. the key difference is that our work produces weakly labeled affective events by prompting pre-trained language models, which alleviates the computational and practical problems of conventional pattern matching over a text corpus. pretrained language models such as @xcite and @xcite have been shown to learn diverse world knowledge @xcite @xcite . researchers have studied how to prompt language models to transfer their knowledge to downstream tasks, including prompt-based fine-tuning, automatic prompt search, and discrete/continuous prompt optimization @xcite @xcite . our work differs in several aspects. essentially, our approach utilizes co-prompts to elicit multiple types of information (views) that are independent and complementary to each other. this is significantly different from prior work that used a single prompt or an ensemble of prompts that seek the same type of information. our work is also related to recent work on data augmentation using language models. for example, anaby-tavor et al. (2020) proposed a method, lambda, that first fine-tunes gpt-2 over labeled data and then synthesizes weakly-labeled data. @xcite proposed a similar but unified approach for pretrained transformer-based language models. @xcite fine-tuned two different generative language models to generate questions and answers separately for reading comprehension. most of these works require fine-tuning the language models while our approach does not. ## acquiring affective events with multiple view co-prompting our research aims to automatically generate labeled affective events to improve classifiers because gold data for affective event classification is only available in limited quantities. automated methods for data generation offer a cost-effective and practical solution for improving the performance of affective event classifiers, and also could be used to rapidly acquire training data for new domains or text genres. figure 1 shows the flowchart for our approach. the process begins with a modest amount of "seed" data consisting of gold labeled affective events. the first step (event generation) uses a language model prompt to elicit events that are associated with each seed event. the second step (polarity labeling) assigns a polarity label to each new event using co-prompting to assess polarity from two independent views of the event. given an event e, the associated event view considers the affective polarities of labeled events that co-occur with e during event generation. the emotion view considers the affective polarities of emotion words that are generated by an emotion prompt given the event e. polarity scores produced from these views are then combined to assign an affective polarity label to the event e. this process repeats in an iterative fashion, where the newly labeled events are used to discover more affective events in the next cycle. the process ends when no new events are generated or a maximum number of iterations is reached. ## limitations we presented a method to automatically generate and label affective events by co-prompting with large language models. the data generation process does not involve creating or training new language models. there are some limitations to our approach. one limitation is that language models are not guaranteed to generate truthful or sensible information, which could introduce noisy information to our model. for example, we observed that the emotion prompt sometimes generates highly unlikely polarity labels for some events. language models can also produce biased results, which could introduce biased information to our model. another limitation is that it may be non-trivial for researchers who want to apply our method to other nlp problems to design prompts that are effective for their task. we believe that this method should be fairly general, but it has not yet been evaluated for other tasks. lastly, our method requires a moderate amount of computational resources, including gpu cards with substantial memory and access to large language models. as a result, groups with limited resources might find our method too computationally intensive.