Ideology Takes Multiple Looks: A High-Quality Dataset for Multifaceted Ideology Detection (2023)

Abstract: Ideology detection (ID) is important for gaining insights about peoples’ opinions and stances on our world and society, which can find many applications in politics, economics and social sciences. It is not uncommon that a piece of text can contain descriptions of various issues. It is also widely accepted that a person can take different ideological stances in different facets. However, existing datasets for the ID task only label a text as ideologically left- or right-leaning as a whole, regardless whether the text containing one or more different issues. Moreover, most prior work annotates texts from data resources with known ideological bias through distant supervision approaches, which may result in many false labels. With some theoretical help from social sciences, this work first designs an ideological schema containing five domains and twelve facets for a new multifaceted ideology detection (MID) task to provide a more complete and delicate description of ideology. We construct a MITweet dataset for the MID task, which contains 12,594 English Twitter posts, each annotated with a Relevance and an Ideology label for all twelve facets. We also design and test a few of strong baselines for the MID task under in-topic and cross-topic settings, which can serve as benchmarks for further research.

## introduction ideologies are the collection of beliefs or philosophies attributed to a particular social group or class @xcite . they shape how we see the world and interact with each other. in this information age, social media has rapidly developed, allowing people to express their thoughts and opinions online, which offer strong cues about their ideologies @xcite authors source size annotated manually multifaceted iyyer et al. (2014) books 3,412 preoţiuc-pietro et al. 2017 twitter 4,833,133 @xcite news articles 120,000 @xcite news articles 1,273 @xcite news articles 754,000 @xcite news articles 34,737 @xcite news articles 2,331,552 @xcite twitter 37,560 mitweet twitter 12,594 it is widely accepted that individuals or groups can take different ideological stances in different facets @xcite . motivated by this, many efforts have been devoted to characterizing ideology from various facets @xcite @xcite . from a computational linguistics viewpoint, some texts may contain descriptions of different issues and reflect the author's ideology from various aspects. as shown in @xcite , the authors convey their multiple ideologies by expressing opinions on topics they care about. therefore, it is necessary to detect the ideology of texts from multiple facets, so as to provide a more comprehensive and nuanced picture for further sociological research. in order to facilitate progress on ideology detection, a number of annotated datasets have been history books will have a wonderfully detailed account of nearly every minute leading up to russia's invasion of ukraine. and a student will ask the teacher one day: if we knew all of this for so long in such detail, why did we do little more than take notes for history? i believe in both bodily autonomy and the sanctity of life. in some situations, abortion is necessary. being pro-life requires being pro-choice. overturning roe v wade was one small vote for a group of mostly menand one giant setback for human rights. released, as shown in in an effort to address these issues, in this work, we first design a multifaceted ideology schema to provide a more complete and delicate description of ideology, by considering traditional ideology division and emerging social topics. as shown in figure 1 (a), the schema covers five orthogonal domains, including politics, economy, culture, diplomacy and society, under which, there are a total of twelve facets. we also define the ideological attributes for the left-and right-leaning of each facet to achieve a more detailed illustration of ideology. based on the new schema, we construct a new high-quality dataset, mitweet, for a new multifaceted ideology detection (mid) task. the dataset contains 12,594 english twitter posts, each manually annotated with a relevance (and an ideology) label for all twelve facets. as the examples in figure 1 (b), mitweet makes up the absence of comprehensive annotation by annotating ideological labels for multiple facets involved in texts. in addition, mitweet covers 14 highly controversial topics in recent years (e.g., abortion, covid-19 and russo-ukrainian war), which can facilitate research in cross-topic mid. using mitweet as the benchmark, we develop strong baselines for the new mid task based on several widely-used pre-trained language models. we conduct thorough experiments and analysis under both in-topic and cross-topic settings. experiment results show that: (1) bertweet @xcite performs the best on overall metrics for both subtasks of mid. (2) the performance of bertweet can be further improved by using indicators as part of input, which is detected with the weighted log-odds-ratio technique with informed dirichlet priors @xcite . (3) crosstopic setting is quite challenging and the achieved performances are far from promising. ## multifaceted ideology to accurately measure the ideology of an individual or group, sociologists have designed a series of scales to categorize political attitudes based on one or more facets. @xcite developed ten stance scales that encompass attitudes towards birth control, the death penalty, censorship, communism, evolution, law, patriotism, theism, criminals, and war. by carefully analyzing the results of the popular test, he was able to condense the ten scales into three distinct ideological facets: religious, humane, and ethnic. soon after, @xcite classified political ideology into "radicalism" and "conservatism". to demonstrate the inadequacy of the above research, @xcite proposed a dual-axis model of ideology based on the principles of freedom and equality. this model is similar to the nolan chart @xcite , a political diagram that uses two facets to divide people's ideological positions into four different quadrants. the horizontal axis represents "economic freedom", while the vertical axis represents "individual freedom". ideologies located in different quadrants are categorized as modern liberalism, conservatism, libertarianism, and nationalism. these prior work proposed different facets for dividing ideologies and laid the theoretical foundation for multifaceted ideology, but a core problem is that they usually study specific domains, such as politics or economics, and different facets may overlap or have ambiguous boundaries. with the development of society, the study of multifaceted ideology and political spectrum has brought attention to new issues, including anti-war movements, women's rights, environmental protection, animal welfare and other related concerns. @xcite argued that color-blind racism is a form of covert and highly institutionalized discrimination, which differs from the more overt forms of racism seen during slavery and legal segregation. therefore, colorblindness serves as the prevailing ideological support for a historically unique form of structural white supremacy known as color-blind racism. @xcite argued that defining gender ideology as a one-dimensional construct that ranges from traditional to egalitarian is problematic in contemporary society. they proposed an alternative framework that considers the multidimensionality of gender ideologies and identified five ideology profiles: egalitarian, essentialist egalitarian, intensive parenting, moderately traditional, and traditional. during the covid-19 pandemic, the far-right ideology has taken on new characteristics, including a disregard for scientific expertise, distrust of news media, and allegiance to trump-christian nationalism @xcite . these new social issues call for a flexible ideological evaluation system, which can fully explain contemporary issues in various social fields and describe the existing variation in ideology from multiple facets. by considering previous work and emerging social issues, we put forth a new multifaceted ideology schema (see figure 1(a) , detailed in appendix a). firstly, our multifaceted ideology schema divides ideology into five orthogonal domains to ensure that each domain is independent and has clear boundaries. secondly, the multifaceted ideology schema now incorporates new contemporary issues. for instance, the facet "cultural value" is linked to the anti-abortion movements, the facet "justice orientation" is associated with the black lives matter movement, and the facet "military force" is connected to peace-war issues. finally, our multifaceted ideology schema covers twelve facets, each is defined with ideological attributes for the left-and right-leaning, providing a more complete and delicate description of ideology. ## the mitweet dataset based on the proposed multifaceted ideology schema as described in section 2, we annotate tweets manually to build mitweet. in this section, we describe how the data is collected, the design of annotation process, dataset statistics and the methods to ensure the quality of annotation. ## conclusion in this paper, we have presented a new multifaceted ideology schema covering five domains and twelve facets to provide a more complete and delicate evaluation system for ideology. based on the schema, we have constructed a high-quality dataset, mitweet, as a benchmark for a new mid task. experiments show that the mid task is quite challenging, especially under cross-topic setting, which requires the knowledge transfer capability of models. we believe that this work has the potential to positively impact both the research and the applications of ideology detection.