Learning affective meanings that derives the social behavior using Bidirectional Encoder Representations from Transformers

1 INTRODUCTION . Consider talking to your mentor for some advice about how to behave with your colleague . Your mentor probably starts asking questions about the culture in the workspace and may continue asking about the identity of the person . These questions could be about institutional constraints such as being the manager , or they could be personality sentiment such as being nice or active . Knowing this information , your mentor may have some initial recommendations , but you adapt your behavior after observing the reactions from the colleague . This is a descriptive scenario for a daily interaction . Affect Control Theory ( ACT ) quantifies the variables in this scenario and produces equations for simulating human behavior in social interactions . ACT was introduced in the 1970s ( Heise , 1977 ) and has been validated in more than 100 social science projects ( Robinson & Smith-Lovin , 2018 ) . ACT has been used in interdisciplinary applications such as Human-Computer Interactions ( Robillard & Hoey , 2018 ) , finding how language cultures affect social response ( Kriegel et al. , 2017 ) , and modeling identities and behaviors within groups ( Rogers & Smith-Lovin , 2019 ) . More recently , Mostafavi ( 2021 ) introduced ACT to estimate and track emotional states during online messaging . For example , chatbots can use the ACT framework to understand the emotional state of the customer in real time and adapt their behavior accordingly . While the potential uses of ACT are understanding emotional changes during online messaging Mostafavi ( 2021 ) , real life applications are limited due to the vocabulary size of affective dictionaries . ACT uses a three-dimensional affective meaning space as a quantified form of sentiments ( Heise , 1977 ) . ACT uses Evaluation [ good vs. bad ] , Potency [ powerful vs. powerless ] , and Activity [ active vs. passive ] ( EPA ) space introduced by Osgood et al . ( 1957 ) as a semantic differential form of affective meaning . These affective lexicons represent the word of interest within cultural and social boundaries ( Robinson & Smith-Lovin , 1992 ) . Fontaine et al . ( 2007 ) found that EPA scores are the first three principal components after reducing dimensionality on 144 features representing the main components of emotions . Historically , surveys are used to quantify the affective meanings within a cultural group . To compensate for unreliability in the survey , most EPA surveys are designed so that each word is scored by at least 25 different participants . Thus , finding the affective meaning for 5000 words requires over 125,000 ratings and 400 hours of respondent time ( Heise , 2010 ) . Because ACT also requires EPA estimates for social interactions , EPA surveys must also include participants to score a large number of additional scenarios . Due to the high cost and time required , most EPA surveys have been focused on a small number of words which has limited the applicability of ACT . As an alternative to conducting new surveys , researchers have tried supervised ( Mostafavi & Porter , 2021 ; Li et al. , 2017 ) and semi-supervised ( Alhothali & Hoey , 2017 ) methods on shallow word-embeddings to find affective lexicons . However , their performance on the activity and potency dimensions is poor . In this paper , we discuss the main limitations for the applicability of shallow word-embeddings . In this work , we use deep sentence-embedding to consider contexual aspects of concepts in social events . For that purpose , we show how to generate a contextual data-set describing social events to train and test a deep neural network . We use Bidirectional Encoder Representations from Transformers ( BERT ) embedding tuned for finding affective lexicons . The performance of our approach achieves state-of-the-art accuracy in estimating affective lexicons . 2 BACKGROUND . In this section , we start with a review of ACT and how it uses sentiment of social characters to model interactions . Then we review related works to estimate affective meanings from corpus and discuss their main limitations . Finally , we briefly review the BERT model and how we can use this model as a deep embedding space . 2.1 AFFECT CONTROL THEORY . According to ACT , every concept representing interactions is quantified in EPA space . The baseline EPA representation of words is known as sentiments . Figure 1 visualizes EPA representation of some sample words . In this plot , we can observe that “ suicidal ” and “ nervous ” both have bad , powerless , and passive meanings but “ suicidal ” is more negative in all three dimensions . On the other hand , “ happy ” is a pleasant , powerful , and active word . Note the range of EPA ratings is from -4.3 to 4.3 . ACT considers social interactions or events that include an actor that behaves toward an object . Extracting the Actor-Behavior-Object ( ABO ) components of an event is the first step in modeling interactions ( Heise , 2010 ) . Actor/Object in an event has an identity such as “ baby ” or “ boss ” . In some cases , the characteristics of an actor/object is part of the identity . For example , the identity of a person is “ nervous boss ” . In these cases , modifiers are amalgamated to their identities . Figure . 1 is a visualization of sample words in EPA space . It uses symbols to represent that a concept is evaluated as identity , behavior , or modifier . According to ACT theory , social events make impressions on ABO characters ( Robinson & SmithLovin , 1999 ) . Consider an example of observing “ a bossy employer argues with an employee ” . This observation leads people to evaluate both the actor and the object of the interaction are less pleasant than initial thoughts . They may also feel the employer is more powerful , and the employee is more powerless than their baseline sentiments after observing this event . Being more pleasant/powerful is impression of observing and event and it means higher value in evaluation/potency . If actor/object behaves as expected , then impression of identity does not change far from baseline , but if actor/object does something unexpected , then a large change from the baseline is expected . Deflection is the euclidean distance between the baseline sentiments of ABO characters and their impressions following an event . If the impression of an ABO event is close to initial sentiments , deflection is small , and it gets bigger when the impression of the event leaves the initial sentiments . ACT discusses that minimizing deflection is the driving force in human activity . Highly deflecting events create social and physiological distress ( Goldstein , 1989 ) . For example , if a grandmother fights with her grandchild , the grandmother and the grandchild feel distressed and prefer to do something . We may expect one side to take an action , like apologize , to bring the impression of their identities back to where they views themselves in the society . This highly deflected event is very different from two soldiers fighting in a battle . The soldiers are supposed to fight with enemies in battle , so they may not feel social pressure to change their behavior . Heise ( 2013 ) developed a software called INTERACT . It simulates sequential interactions between two identities and finds the behavior that minimizes the deflection . It can also predict attributes and emotions during the interaction . Consider the following set of events/interactions that we simulate using INTERACT , 1 . Employee greets bossy employer . 2 . Bossy employer asks employee . 3 . Employee replies to bossy employer . 4 . Bossy employer argues with employee . 5 . Employee listens to / disobeys bossy employer . The visualization in Figure 2 shows how the impression of actor/object ’ s identity changed based on sequential interactions . Let ’ s focus on evaluation dimension for the employer . Employer has a negative baseline evaluation but after observing the first two interactions , it increases . The first two interactions include positively rated behaviors . After second interaction , impression of the employer ’ s identity is positively evaluated and so the next positively evaluated action , replies to , does not move it substantially . A positive behavior is expected from a positive identity . However , in the fourth interaction , the employer is evaluated to have an unpleasant identity after doing a negative behavior , argue with . For the fifth event , we have shown how the impression of different actions by the employee has significantly moved the states for both the actor and the object . The sequential interactions discussed here are similar to our mentorship example discussed the beginning . It shows how understanding the interaction dynamic can make us predict the consequences of our behavior . ACT has rules to describe how impression of an events changes affective meaning of ABO characters . ACT uses either mathematical equations or descriptive forms to discuss these rules . The following two descriptive forms show how the identity of the actor is impressed by some events , • Actors seem nice when they behave in a positive way toward others . This describes morality fact in ACT literature . Observing the evaluation dimension for the actors after he greets the object , we can find this behavior resulted in an impression of being nicer ( getting larger evaluation ) comparing to the state in the last step . • Active behaviors make the actors seem more active . Observing the boss ’ s activity , he is considered more active after he argues with [ active behavior ] the employee . As we have seen in the descriptive forms , events can change the impression of ABO characters . They move them toward or away from their currentor initial sentiments . To formulate the process in mathematical space , we briefly review the quantification process using surveys . The first step is quantifying the sentiments that are introduced as identity , behavior , and modifier . For this purpose , at least 25 participants rate words of interest in EPA space ( Heise , 2010 ) . In this survey , the participants rate how they feel about an identity/behavior/modifier such as “ employer ” . After aggregating the sentiment surveys , every concept is assigned to its baseline affective meaning . The next step is identifying contributing facts that derive the impressions of events ( Robinson & Smith-Lovin , 1999 ) . For this purpose , the participants rate ABO characters again after observing a set of events . For example , participants rate affective meaning of “ employee ” , “ greet ” , and “ employer ” after observing “ the employee greets the employer ” . As we discussed earlier , the ratings of ABO could be different from the initial basements . ACT uses regression models , known as impression change equations , to estimate these changes ( Heise , 2013 ) . Let X = [ Ae , Ap , Aa , Be , Bp , Ba , Oe , Op , Oa ] T represent the EPA values/sentiments of an ABO triple , where { A , B , O } represent the ABO characters and { e , p , a } the EPA components . Consider further the two-way interactions X2 = [ AeBe , AeOe , AeBa , . . . AaOa ] T and three-way interactions X3 = [ AeBeOe , AeBeOp , AeBeOa , . . . AaBaOa ] T . The basic structure of an impression change equation is the linear model X ′ = αX + βX2 + γX3 ( 1 ) where α , β , and γ are coefficient vectors and X ′ represent the resulting impression after the event . Modifiers can incorporated prior to impression change by changing the baseline values/sentiments ( e.g . bossy employer ) . Averett & Heise ( 1987 ) defined amalgamation equations similar to ( 2 ) to find the sentiments for an identity with a modifier . A = ρ+ θM + ψI , ( 2 ) ρ = [ −0.17 −0.18 0 ] , θ = [ 0.62 −0.14 −0.18 −0.11 0.63 0 0 0 0.61 ] , ψ = [ 0.50 0 0 0 0.56 0.07 0 −0.05 0.60 ] . where , A , M , I , represent affective meaning of actor ’ s identity , modifier , baseline identity and ρ , θ , ψ are vector of intercepts and coefficient matrices . Equation ( 2 ) is a weighted average of the evaluation for the modifier and the identity .

Summary: This paper proposes a new sentiment representation method by using Affect Control Theory (ACT) and BERT model. Using ACT could capture the social interactions and sentiment changes. The main contribution of this paper is the usage of ACT and the whole process. According to the experimental results, the BERT model shows good performance.