Social Media Unrest Prediction during the COVID -19 Pandemic: Neural Implicit Motive Pattern Recognition as Psychometric Signs of Severe Crises (2020)

Abstract: The COVID-19 pandemic has caused international social tension and unrest. Besides the crisis itself, there are growing signs of rising conflict potential of societies around the world. Indicators of global mood changes are hard to detect and direct questionnaires suffer from social desirability biases. However, so-called implicit methods can reveal humans intrinsic desires from e.g. social media texts. We present psychologically validated social unrest predictors and replicate scalable and automated predictions, setting a new state of the art on a recent German shared task dataset. We employ this model to investigate a change of language towards social unrest during the COVID-19 pandemic by comparing established psychological predictors on samples of tweets from spring 2019 with spring 2020. The results show a significant increase of the conflict indicating psychometrics. With this work, we demonstrate the applicability of automated NLP-based approaches to quantitative psychological research.

## introduction the covid-19 pandemic and the reactions to it have led to growing social tensions. guitérrez-romero (2020) studied the effects of social distancing and lockdowns on riots, violence against civilians, and food-related conflicts in 24 african countries. the author found that the risk of riots and violence have increased due to lockdowns. resistance against national health regulations such as the duty to wear masks are partially met with resistance by movements such as anti-maskers or anti-obligation demonstrations. 1 even anti-democratic alterations of e.g. services offered by the us postal service (usps) of delivering mail-in ballots for the us presidential elections 2020, which are essential for social distancing measures amidst the pandemic, are being utilized amidst this international crisis and foster social unrest and potential outbursts of violence, civil disobedience or uprisings. 2 social media has become an important reflection of nationally and internationally discussed topics, and is a predictor of e.g. stock markets, disease outbreaks or political elections @xcite . the majority of human-produced data exists in textual form and broadly in social media and thus, an the investigation of social unrest and conflict situations from social media becomes a worthwhile application area for natural language processing (nlp) problem @xcite . when speaking about such global phenomena such as a rise in international social unrest and possible occurrences of conflict reflected in text, the detection of specific keywords or utterances have not been successful in past research. @xcite utilized laten dirichlet allocation (lda, @xcite ) topic modelling on war-related newspaper items and were not able to improve predictability from other multi-factor models that take into account e.g. gdp figures, mountainous terrain or ethnic polarization. furthermore, @xcite showed that news reports on possible war situations alone did not function as good predictors but identified sharp frequency increases before war emerged, possibly helping with just-in-time safety measures but likely failing to avoid war situations altogether. alternatively, the risks of escalation could be determined based on politician's personalities and the current mood and tone of utterances @xcite . however, intrinsic desires and personality can hardly be measured directly (see section 3). intrinsic or subconscious desires and motivation would more likely correlate with personalities, tone, and thus possibly social unrest. we hypothesize that the frequency of social unrest predictors have significantly changed in social media textual data during the covid-19 pandemic drawn from the twitter 1 percent stream foot_0 in early 2019 and 2020, whilst linguistic features stay comparably stable and unchanged. with this, we aim to demonstrate a possible transition from laborsome manual psychological research to automated nlp approaches. after presenting and discussing related work in section 2, we will first introduce the concept of implicit motives and self-regulating levels in more details in section 3 and the social unrest predictors thereafter in section 4. the data utilized for experiments is described in section 5 and the methodology in section 6. thereafter, we will present the results in section 7 and discuss their impacts in section 8. lastly, we will draw a conclusion in section 9. ## related work conflict predictions from natural language are rarely encountered applications and have mainly been about content analysis and less about crowd psychology. @xcite used one-to-x analogy reasoning based on word embeddings for predicting previous armed conflict situations from printed news. @xcite performed named entity recognition (ner) and extracted events via hidden markov models (hmm) and neural networks, which were combined with human intelligence reports to identify current global areas of conflicts, that, in turn, were utilized mainly for world map visualizations. investigation of personality traits has mainly been focussing on so-called explicit methods. for these, questionnaires are filled out either by interviewers, through observations, or directly by participants. one of the most broadly utilized psychometrics is the big five inventory, even though its validity is controversial @xcite . the five-factory theory of personality (later named big five) identifies five personality traits, namely openness to experiences, conscientiousness, extraversion, agreeableness and neuroticism @xcite . this big five inventory was utilized by @xcite for analyzing these five traits of filipino speakers. some studies perform natural language processing (nlp) for investigating personality traits. @xcite utilized an attention mechanism for deciding upon important parts of an instance when assigning the five-factor inventory classes. the myers-briggs type indicator (mbti) is a broadly utilized adaption of the big five inventory, which @xcite employed for asserting the personality traits within tweets. 4 the research field of psychology has moved further towards automated language assertions during the past years. one standard methodology is the utilization of the tool linguistic inquiry and word count (liwc), developed by @xcite . the german version of liwc was developed by @xcite . it includes 96 target classes, some of which are rather simple linguistic features (word count, words longer than six characters, frequency of punctuation), and psychological categories such as anxiety, familiarity, or occupation. even though the tool appears rather simple from an nlp point of view, it has a long tradition to be utilized for content research in the field of behavioral psychology. studies utilizing liwc have shown that function words are valid predictors for long-term developments such as academic success @xcite . furthermore, it has been shown that liwc correlates with the big five inventory @xcite . importantly, the writing style of people can be considered a trait, as it has shown high stability over time, which means that it is not dependent on one's current mood, the time of day, or other external conditions @xcite . @xcite utilized an implicit motive (see section 3) dictionary approach to automatically determine risks of war outbreaks from different novels and historic documents, identifying widening gaps between the so-called power motive and affiliation motive in near-war situations. overall, the work on automated classification of implicit motive data or the use of nlp for the assertion of psychological traits in general is rather sparse or relies on rather outdated methods, as this application domain can be considered a niche @xcite @xcite . one recent event in this area was the germeval 2020 task 1 on the classification and regression of cognitive and motivational style from text @xcite . the best participating team reached a macro f1 score of 70.40 on the task of classifying implicit motives combined with self-regulating levels, resulting in 30 target classes. however, behavioral outcomes from automatically classified implicit motives have -to our knowledgenot yet been researched. ## implicit motives and self-regulatory levels implicit motives can reveal intrinsic, unconscious human desires, and thus avoid social desirability biases, which usually are present when utilizing direct questionnaires. they originated from the thematic apperception test (tat) by @xcite . participants are confronted with ambiguous images of multiple people that interact with each other as displayed in figure 1 , and are asked to answer four questions: i) who is the main person, ii) what does that person feel? iii) why does the person feel that way, and iv) how does the story end? from these questions, trained psychologists can assign one of five motives: affiliation (a), freedom (f), achievement (l), power (m), and zero (0). the psychologists follow some rules, one being the so-called primacy rule, where the very first identifiable motive determines the whole instance, despite what follows @xcite . these motives have shown to be behavioral predictors and allow for long-term statements of e.g. group dynamics or success @xcite . implicit motives have been broadly utilized in the 1980s but at the cost of laborsome manual annotating processes. it takes about 20 hours of training for an annotator to encode one of the implicit motives. skilled human annotators take up to 50 hours per 100 participants. this costly annotating process has hampered this once-promising psychometric (schultheiss and @xcite . whilst the classification performance of implicit motive models have been explored and achieved high results (e.g. @xcite ), behavioral consequences and mass phenomena from automated labeled textual instances have barely been researched. in addition to the implicit motives, the data set from the germeval20 task 1 comes with so-called levels per textual instance. the levels were developed by @xcite . they describe the self-regulatory enactment in five dimensions. according to @xcite the 1st level is the ability to self-regulate a positive affect, the 2nd is the sensitivity for positive incentives, the 3rd self-regulates a negative affect, the 4th is the sensitivity for negative incentives and the 5th level describes the passive coping with fears. in other words, these levels help to identify the type of the participant's emotional response according to the identified implicit motive. as with many psychometrics, the reliability of implicit motives, and especially their predecessor (the tat) is controversial. one main point of criticism is that implicit motives do not correlate significantly with so-called explicit motives. whilst implicit motives try to measure intrinsic desires indirectly by asking participants associative questions, explicit motives try to measure desires via direct questionnaires. in psychology, reliability means, that personality traits revealed by one measure may not conflict with personality traits measured by other, well-established measures. since the measured desires of implicit and explicit motives generally do not match, the reliability of implicit motives is said to be weak. @xcite explain this lack of reliability and correlation with the fact that explicit implicit motives are by definition of different measurements that can not be directly compared. whilst implicit motives measure intrinsic desires that are subconscious, explicit motives are more influenced by a social expectation bias (i.e. what do participants think is a socially sound and accepted answer to a question) and thus are closer connected to behaviorism. nonetheless, reliability in psychological research demands different observable results of metrics to be coherent @xcite when the descriptions of what a psychometric is supposed to measure matches (i.e. desires). another point of criticism is the way the tat images are selected. they emerge from an empirical study, where participants are shown different images. only when past frequencies of motives are achieved with an image, this image gets added to the available testing stock. with this, however, the very first selected implicit motive images could not have been validated. nowadays, many scholars argue that the amounts of positive evidence legitimize this methodology, but it has yet to be resolved @xcite . figure 1 : exemplary image to be interpreted by participants utilized for the operant motive test (omt). identifiable motives are the affiliation motive (a), the power motive (m), achievement (l), and freedom (f). a 0 represents the unassigned motive @xcite . the transition from natural language to intrinsic desires and motivation is not trivial, as humans do not express intrinsic and unconscious desires unfiltered and directly. as soon as a direct questionnaire is involved, social desirability biases (i.e. thoughts of publicly expected answers) alter an uninfluenced introspection @xcite . such direct questionnaires are called explicit methods, in contrast to implicit methods, such as e.g. the tat and subsequent tests produced by image descriptions. ## social unrest predictors times of severe social unrest are reflected by distinct patterns of implicit motives and linguistic features. @xcite surveyed multiple prior studies, identifying three main predictors: responsibility, activity inhibition, and integrative complexity, displayed in activity inhibition is reflected, according to by @xcite as the frequency of "not" and "-n't" contradictions in tat or other verbal texts. activity inhibition functions as motivational and emotional regulation. the authors identified a negative correlation between activity inhibition and male alcohol consumption. combined with a high power motive (m) and low affiliation motive (a), subsequent research by mcclelland and his colleagues revealed a so-called leadership motive pattern (lmp) @xcite . the higher this lmp, the more responsible leaders act. as for integrative complexity it was observed, that the lower the frequency of utterances in accordance to the 7-point score was (see ## training and experimental data for testing the proposed hypothesis (section 1), we first train a classification model and utilize this model for testing social network textual data. in this section, we will describe the two different data sources for training and the experiments. ## methodology for implicit motive classification social unrest prediction for constructing a model of sufficient quality to test our hypothesis, we follow johannßen and biemann (2019) and train a long short-term memory network (lstm, @xcite ) combined with an attention mechanism. an lstm is a special type of recurrent neural network (rnn). an rnn not only has connections between units from layer to layer but also between units of the same layer. furthermore an lstm also has a mechanism called the forget gate, allowing the structure to determine which to keep and which information to forget during the training process. the attention mechanism @xcite can capture the intermediate importance of algorithmic decisions made by the network. it can be employed for enhanced results but also investigated for researching algorithmic decisions. however, it is debated upon, whether this algorithmic importance can serve as an explanation. even though oftentimes, the algorithmic importance is correlated with an explanation for the task (i.e. does a model for image recognition of animals look at the animals or the backgrounds of the images?), there are cases, where algorithmic importance and explanation for the task differ @xcite ). since automatically labeling implicit motives is a sequential problem revolving around identifying the first verbal enactment of a motive (see section 3, we decided to employ a bi-lstm with an attention mechanism @xcite . we decided against additional features such as part of speech (pos) tags or liwc features like in our previous work @xcite , as we did not reach the best results with these additional features. the maximum token length was set to 20, as determined by preliminary experiments @xcite , and reflects the primacy rule of the implicit motive theory described in section 3. the average answer length of the training data set was 22 tokens (see section 5). with this decision to limit the considered tokens, we aim to closely replicate the implicit motive coding practices manually performed by trained psychologists @xcite . accordingly, it is preferable to assign the 0 motive (i.e. no clear motive could be identified) than to falsely assign a motive that is not the very first one in the sequence. some standard pre-processing steps were applied to reduce noise, which was to remove the natural language toolkit (nltk) german corpus stop words foot_5 , to lowercase the text, remove numbers, normalize special german letters (i.e. umlaute). emojis were removed as well, since twitter offers a selection of a 3,348 emojis foot_6 , that in turn mainly do not capture sufficient informational gain per textual answer for the task at hand. to remove stop words has to be an informed choice when it comes to performing nlp on psychological textual data. for example, function words are said to predict academic success (see section 2). however, during our experiments, we saw an increase in model performance when stop words were removed. after the training, we utilize the model on the two sampled data sets described in subsection 5.2. according to our hypothesis in section 1 and following the theories in section 4, investigate the frequency of the power motive with the self-regulatory level 4, which we expect to be higher. at the same time, we will also analyze the other motives and levels to see which ones are now less frequent and to what extent. furthermore, we compare different linguistic features and statistics from 2019 to 2020 to see, if any of these show differences that might indicate possible biases in the data. our bi-lstm model was set to be trained within 3 epochs and with a batch size of 32 instances. the model was constructed having 3 hidden layers and utilized pre-trained fasttext embeddings @xcite , as this character-based or word fragment-based language representation has shown to be less prone to noisy data and words that have not been observed yet like e.g. spelling mistakes or slang -both often observable in social media data. the fasttext embeddings had 300 dimensions and were trained on a twitter corpus, ideally matching the task at hand. 10 explorative experiments with different parameter combinations have shown that a drop-out rate of .3 and step width of .001 produced good results. the cross-entropy loss was reduced rather quickly and oscillated at 1.1 when we stopped training early during the second epoch. after each epoch, the model was evaluated on a separate development test. after the training was finished, the model was tested once on the germeval20 task 1 test data and with the official evaluation script. this provides the chance to compare the achieved results with the best-participating team. schütze et al. (2020) achieved a macro f1 score of 70.40, which our bi-lstm model was able to outperform with an f1 score of 74.08, setting a new state of the art on this dataset. ## results after having trained the bi-lstm model and sampled the experimental data, we will describe the results and findings of the conducted twitter covid-19 experiments in this section. an overview of all results is displayed in to investigate the main predictor for social unrest activity inhibition (see section 4), the power motive (m) in combination with level 4 was counted. the self-regulatory level 4 describes the sensitivity for negative incentives (see section 3). these measures are collected for all four data sets. our bi-lstm model assigned power 4 in 33.76% of all cases for the twitter sample from march to may of 2019, making this the most frequent label. however, for the data sample from 2020, power 4 is as frequent as note that most motives, levels, and statistical values stay constant. however, power 4 is more frequent, whilst the freedom motive is less. as the linguistic statistic metrics stay relatively stable, this indicates no observable sampling bias. the affiliation motive (a) is barely classified, covering only 2% (2019) and 1.89% (2020) of all instances. the slight decrease is not statistically significant (@xmath0.05). the frequency of self-regulatory level 4 is elevated by 6.7%. the whole of all assigned power motive labels has only risen by 3.64%, both having risen less than the combination of the power motive and level 4 combined. the strongest decline in frequency can be measured for the freedom motive with -12.63%. the other motives of affiliation, achievement, and null have barely changed in comparison to 2019 with 2020. the same holds for the average amounts of words per sentence, verbs, adjectives, and words containing at least 6 letters, all of which have barely changed, not indicating sampling biases. an overview of the class frequencies is provided in since both, responsibility and integrative complexity can only be measured by employing a specific tat and a questionnaire, which would have to be performed with each twitter user, we can only investigate activity inhibition as a combination of the power motive with the self-regulatory level 4. however, we will review some psychological liwc categories, that follow a close description as the five categories ## discussion we hypothesized that the social unrest predictors by @xcite , namely activity inhibition, responsibility, and integrative complexity are automatable and reveal changes in natural language and signs of social unrest observable through the use of social media textual data connected to the covid-19 pandemic. the main research objective of this work is to find novel approaches to automatically provide the community with red flags for growing tensions and signs of social unrest via social media textual data. for this, activity inhibition is the main predictor. it consists of a distinct shift in implicit motives. it is present when the frequency of the power motive with the self-regulating level 4 (sensitivity for negative incentives, see section 3) is elevated and the affiliation motive is suppressed -even though @xcite did not find clear evidence of the latter. the comparable rise by 10.97% (@xmath2.01) is an indicator of the social tension of covid-19 related social media posts. since other linguistic statistics, such as the average amounts of adjectives, verbs, words per sentence, or words containing at least 6 letters have barely changed, this indicates that the measurable differences in social unrest predictors are content-based and not due to linguistic biases. it is remarkable, that whilst the power motive has been labeled more frequently, the frequency of the labeled freedom motive has declined by -12.63% from 2019 to 2020. this freedom motive has barely been researched yet but has a close connection to the power motive. whilst power-motivated individuals desire control over their fellow humans and their direct surrounding for the sake of control, freedommotivated individuals seek to express themselves and want to avoid any restraining factors. motives are said to be rather stable but can change over time @xcite . this change in motive direction could indicate a roughening of verbal textual content and interpersonal communication. example utterances classified as freedom and power from 2019 compared with 2020 are displayed in the change of responsibility, as reflected in liwc categories, retreated by roughly 30% from 2019 to 2020. this responsibility indicates a personal involvement in topics and decisions, that we feel are relevant for our surroundings. if this involvement diminishes, our interest in participating in constructive solutions to problems does as well. m 'rt @fraulavendel: ist es wahr dass schulleitungen den schüler * innen drohen f rt @uteweber: nach einem relativ unfeierlichen, regionalen offline-tag aufs sofa sinken, wie von der tarantel gestochen aufspringen und zur... a weltbestseller "p.s. ich liebe dich" bekommt einen zweiten teil https://t.co/9ifl5crnap -------translation -----m 'rt @fraulavendel: is it true that principals threatens students f rt @uteweber: after a relatively un-celebrational, regional offline day, as bitten by a tarantula jumping up a world best-selling book "p.s. ich liebe dich" gets a second part https://t.co/9ifl5crnap m corona-regeln im saarland sind zum teil absurd und unverhältnismäßig f rt @kattascha: in den usa bekommen viele menschen keine lohnfortzahlung im krankheitsfall. das bedeutet: selbst bei verdacht auf #covid19 w... a wer einen discord-server sucht, um entspannt mit seinen kollegen zu zocken oder gemeinsam abzuhängen ist hier genau... -------translation ----- m the corona rules for the saarland are partially absurd and disproportionate f rt @kattascha: in the us a lot of people don't receive continued pay in case of illness. that means: even in case of suspected #covid19 a whoever is looking for a discord server for enjoyably game with their colleagues or chill together, is in the right place... ## conclusion and outlook with this work, we conducted a first attempt at automating psychometrics for investigating social unrest in social media textual data. the bi-lstm model combined with an attention mechanism of this work achieved an f1 score of 74.08 on 30 target classes, making it state of the art on a respective recent shared task dataset. with this model, we measured a statistically significant rise in the power motive with selfregulating level 4, which reflects the social unrest predictor of activity inhibition in the direct comparison of the samples from march to may of 2019 vs. 2020. furthermore, we investigated responsibility, which shows significant reductions during the covid-19 pandemic, hinting at negative outcomes of interpersonal and verbal communication on the social media platform twitter. this first approach most likely does not qualify for a real-world social prediction system. predictions of such a system can not yet be reliable enough for deriving necessary actions from them. on the upside, implicit motives do not only qualify for examining general socio-economic tensions, but can be applied on an individual or small group scale. as an example, detecting tensions within a small group can help to shape the group and guiding it into a better fit. furthermore, we advocate for combining implicit motives with sufficiently many complementary psychometrics and content-based analysis e.g. sentiment analysis, topic modeling, or emotion detection. besides those combinations with other information sources for future work, different sampling approaches and larger data set sizes should be utilized for reproducing findings and research correlations with other social unrest predictors and indicators. in this work, we have made the first steps towards understanding the automation of psychological findings. since only 5,000 samples were drawn from a single social network platform, we advocate for broadening this approach to include many more samples from wider time windows paired with mixing the data sources. in addition to that, deeper investigations into the linguistic variances between times of so-called social unrest and more peaceful times should be performed, as those could reveal patterns and characteristics of time-specific utterances. even though this work is only introductory, the observed correlations and social unrest patterns are in line with an intuitive assumption of how language in social media data changes amid a pandemic. future work arises in the application of this methodology on other events and crises, eventually providing a quantitative basis for implicit motive research.