Question: What frames are used to quantify the amount of agency attributed to a target group?
Evidence: While paragraph-level valence analysis is quick and simple, it is sometimes too coarse because we aim to understand the sentiment directed towards the target group, not just nearby in the text. For example, suppose the target group is named “B". A sentence such as “A violently attacked B" would likely have extremely negative valence, but the writer may not feel negatively towards the victim, “B".
We address this by using BIBREF22's Connotation Frames Lexicon, which contains rich annotations for 900 English verbs (BIBREF22). Among other things, for each verb, the Connotation Frames Lexicon provides scores (ranging from -0.87 to 0.8) for the writer's perspective towards the verb's subject and object. In the example above for the verb attack, the lexicon lists the writer's perspective towards the subject “A", the attacker, as -0.6 (strongly negative) and the object “B" as 0.23 (weakly positive).
We extract all subject-verb-object tuples containing at least one target group label using the Spacy dependency parser . For each subject and object, we capture the noun and the modifying adjectives, as group labels (such as gay) can often take either nominal or adjectival forms. For each tuple, we use the connotation frame lexicon to determine the writer's perspective either towards the subject if the group label appears in the subject noun phrase, or perspective towards the object if the label appears in the object noun phrase. We then average perspective scores over all tuples.
As in Section SECREF6, we use Connotation Frames to quantify the amount of agency attributed to a target group. We use BIBREF25's extension of Connotation Frames for agency BIBREF25. Following BIBREF25's interpretation, entities with high agency exert a high degree of control over their own decisions and are active decision-makers, while entities with low agency are more passive BIBREF25. This contrast is particularly apparent in example sentences such as X searched for Y and X waited for Y, where the verb searched gives X high agency and waited gives X low agency BIBREF25. Additionally, BIBREF25's released lexicon for agency indicates that the subjects of verbs such as attack and praise are given high agency, while the subjects of doubts, needs, and owes are given low agency BIBREF25.
This lexicon considers the agency attributed to subjects of nearly 2000 transitive and intransitive verbs. To use this lexicon to quantify denial of agency in our corpus, we extract all sentences' head verbs and their corresponding subjects, where the subject noun phrase contains a target group label. Unlike BIBREF22's real-valued Connotation Frames lexicon for perspective, the agency lexicon only provides binary labels, so we calculate the fraction of subject-verb pairs where a subject containing a group label was given high agency by its head verb.
Figure FIGREF34 shows the writer's average perspective (valence) towards noun phrases containing either any LGBTQ labels, gay(s), homosexual(s), or the comparison group American(s) using the Connotation Frames lexicon BIBREF22. Note that the wide variation, particularly for homosexual, is likely due to sparsity, as limiting the connotation frames analysis to verbs' immediate subject and direct object noun phrase dependents (consisting of only determiners, adjectives, and nouns) greatly reduced the amount of data for each year. For example, there were only 39 triples for homosexual in 2015. We thus show results aggregated over five-year intervals, as in Figure FIGREF27.
As with paragraph-level valence, the writer's perspective towards the label homosexual is significantly more negative than towards gay ($ p < 0.001$). Linear regression indicates that perspectives towards noun phrases named by any LGBTQ term, gay, and American have all significantly increased over time ($p < 0.01$). However, the trends are still quite different, as the slopes for gay and all LGBTQ terms are an order of magnitude greater than American ($m =(1.1\pm 0.39)\times 10^{-4}$ for American, $m=(1.4\pm 0.18)\times 10^{-3}$ for all LGBTQ terms, and $m=(1.1\pm 0.22)\times 10^{-3}$ for gay). Furthermore, the writer's perspective towards noun phrases containing homosexual have significantly decreased over time ($p < 0.0001$).
Overall, connotation frames' perspective scores reveal a similar pattern as the paragraph-level valence analysis, where LGBTQ groups overall appear to be more positively evaluated in the New York Times over time. Unlike gay and the aggregated all LGBTQ terms, the label homosexual undergoes pejoration, as homosexual becomes increasingly used when (implicitly) expressing negative attitudes towards LGBTQ people.
To qualitatively analyze how well the connotation frames' lexicon capture the negative evaluation of a target group component of dehumanization, we identify subject-verb-object tuples where the verb indicates that the writer has extremely positive or negative perspective towards either the subject or object. The first two paragraphs in Table TABREF36 were identified among those with the most negative writers' perspectives towards phrases containing LGBTQ labels. The first paragraph (within a quote) uses the phrase any homosexual act as the direct object to the verb committed, which has the effect of framing homosexuality as a crime or other illicit activity. By deeming gay candidates unworthy of the priesthood, the speaker clearly negatively evaluates LGBTQ people. On the opposite end, many of the paragraphs labeled by our method as containing extremely positive perspectives towards phrases containing LGBTQ labels do appear to have positive evaluations of these groups. The second and third paragraphs of Table TABREF36 illustrate this, where the gays are viewed positively for having saved a town, and gay rights advocates are praised for their work.
However, we found several instances where paragraphs seem to have been mislabeled, which are shown in Table TABREF37. In the first paragraph of Table TABREF37, our technique identifies gay marriage as the subject of dependent of the negative-perspective verb harmed, but does not account for the preceding text, which actually reveals that the paragraph contradicts the premise that gay marriage causes harm, and thus does not contain overtly negative evaluations of LGBTQ groups (although this particular example reveals the difficulty of operationalizing this component because ProtectMarriage groups strongly oppose same-sex marriage and may itself have negative evaluations of LGBTQ people). The second example similarly shows that this simple method does not adequately account for various forms of negation, as the positive-perspective verb protect is actually negated. The last example in Table TABREF37 presents a complex case, and it is even qualitatively challenging to determine the writer's perspective towards LGBTQ people. Our method identifies gays as the subject of the verb strengthen, even though the subject should be the gerund allowing gays (into the military), and the lexicon's entry for the writer's perspective towards the subject of of strengthen is a highly positive 0.7. However, the object of this verb is the terrorist organization Al Qaeda; our background knowledge suggests that the capacity to strengthen Al Qaeda would reflect negative perspectives. However, this additional context provided by the rest of the paragraph indicates that the writer is being sarcastic and considers the proposition that gays have any impact on strengthening Al Qaeda to be ridiculous. Finally, the writer emphasizes their own stance in opposition to the Missouri congressional candidate by calling upon common stereotypes of gay people being good at dancing and accessorizing.
Measuring the connotation frames' lexicon perspective scores over verbs' subjects and direct objects cannot leverage as much context or data as measuring valence over paragraphs using the NRC VAD lexicon labeled for 20,000 words. However, this technique can make more fine-grained distinctions regarding the writer's (and institution's) attitudes directed towards LGBTQ people and is not as dramatically impacted by the emotional valence of the topic discussed. However, with both techniques presented, we have difficulties disentangling the journalist's perspective from those expressed by others and simply reported by the journalist. While removing direct quotations may partially address this issue, we deliberately did not remove text from direct quotes or paraphrases. The journalists and newspaper make intentional decisions about what text to include and exclude from quotations, which could still meaningfully represent their perspectives and values BIBREF69.
As in Section 3.1, we use Connotation Frames to quantify the amount of agency attributed to a target group.
Answer: Connotation Frames