We are confident in trusting Language Models for annotating online sexism in political discourse, but are they good?

Authors: Aditi Dutta
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: online sexism detection, political discourse, language model annotation reliability, model confidence estimation

Abstract:
Large Language Models (LLMs) have recently gained popularity for text analysis within the social sciences due to their versatility and context-aware capabilities. The use of prompt-based learning of LLMs has especially increased its application in classification tasks and text annotation of sensitive topics like sexism. While studies have used them for capturing online sexism, not much has been known of their capabilities across lesser-known discourses like that of political discourse, and how the models distinguish between partisan bias to gender bias. In this research, our main contributions could be listed as: i) comparing different LLMs through prompt engineering in their capability of detecting sexism in political discourse; and ii) proposing a new algorithm for capturing the confidence of the LLM predictions in classification tasks. Experimental results demonstrate a clear indication of trigger events that provoke online sexism, and yet no clear advantage of using LLMs while predicting sexism. Surprisingly, the results do not improve with more instructive prompts, but our algorithm proves to be effective in capturing the confidence of each model on their predicted labels.

Decision: reject
Reviewer scores: 5, 6, 6, 5

Reviews and discussion:
<thread>
creation date:
1730617942900

thread_type:
official_review

<thread_object>
creator: reviewer_cagn

<content>
summary:
This paper brings a focus on the task of detecting sexism specifically from political discourse. It includes an assessment of existing LLMs through prompt engineering when it comes to this task and then propose a new algorithm for capturing the confidence of the LLM
predictions in classification tasks. Their methodology illustrates trigger events in text that provoke sexism. Nonetheless they also claim that it provides no clear advantage when predicting sexism.

strengths:
The model covers a thorough analysis of sexism detection on political discourse online using state of the art Large Language Models. For such a niche task, prompt engineering seems to be the right approach but they also put a detailed analysis into the robustness of the prompts they use as well as the calculation of a confidence score based on the outputs yielded by LLMs.

weaknesses:
While this work addresses an important issue in language i.e. sexism detection and also provides a thorough picture of state-of-the-art LLMs when it comes to solving it, the overall novelty of this paper is low. It does not provide any specific understanding of what drives sexism within political narratives and or solutions of how models can be trained to capture such semantic cues in text.

questions:
The central thesis of this paper is understanding and detecting sexism in political discourse, yet it fails to specifically highlight what about political discourse require special investigation than regular text for sexism detection.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730621447406

thread_type:
official_review

<thread_object>
creator: reviewer_etqd

<content>
summary:
The paper investigates how efficient are LLMs in predicting sexism in online discourse. The experiments are conducted on a twitter dataset conducted as part of the study which includes tweets from 22 with location as UK. They use 5 different LLMs and 4 different prompt types to evaluate the sexism prediction on the dataset. In addition, the also introduce a metric for predicting confidence of the models in their predictions using an uncertainty estimation.

strengths:
- The paper uses an extremely imbalanced dataset to predict sexism in online political discourse.However, this is similar to real-world scenario where majority of the texts are non-sexist, making the research question quite interesting. 
- In addition to model comparison, they also perform error analysis to understand the reason of mis-classifications.
- They provide a relatively straight forward method to measure confidence of the models in their predictions.

weaknesses:
Confidence metric:
- A temperature of 0 is used during the confidence estimation. This is practical if you want to evaluate the model on a deterministic response. However, in a practical setting, we do not use low/zero temperature as the models would constraint models diversity. It would be have been more intuitive to see the variation in the metric across different temperatures or at least on the temperature setting shows some level of randomness.
- The number of iteration used is 3, which is probably sufficient given that at temperature 0, the variance in the generated output will be very less. But small variations in the seed values might influence the model predictions. So, I am not entirely sure if the number of iterations would be sufficient for statistically robust confidence. 

Clarity in writing:
- While the paper explores interesting research questions like confidence estimation of LLMs, lack of clarity in writing style seriously affects the readability of the paper. I would suggest not using passive voice in sentences and rewriting the content for better clarity should be done before acceptance.

questions:
- Line 201 says "**four** incidents of 2022 were chosen.. " and line 206-207 says  We attributed the **three** different trigger events to two trigger types", was this a mistake ? Or are the **incidents** different from **events**?
- In table 2, there are 2 tables, one showing "Political trigger event" and the other "Sexist trigger event". I am assuming that the total size of the dataset is the sum of the two and you reported the model performance on the combined version of the dataset. Is my assumption right? In any case, this explanation could benefit from a bit more clarity in writing.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730697944901

thread_type:
official_review

<thread_object>
creator: reviewer_tt8a

<content>
summary:
This paper presents an evaluation of various LLMs in detecting sexism within political discourse, proposing a new algorithm to measure prediction confidence. The authors discuss the limitations of current LLMs in detecting subtle, context-specific sexism through prompt engineering, suggesting that prompt specificity may not enhance model accuracy (as anticipated).

strengths:
-	This paper is centered around a very interesting topic which is highly relevant given the current online landscape.
-	The corpus will be a useful resource for further research on the subject.
-	Clearly scoped tasks and objectives.

weaknesses:
The paper is generally well written, however there are a few points that must be addressed:

- A section describing existing corpora (annotated for sexism) is missing. The authors should include a brief overview of the existing datasets, including their annotation scheme.

- I believe that since the company was bought, what had previously been free access to Twitter’s API, has been removed. Was the data collected before the cutoff date (~March 2023)?

- At the beginning of Section 3.1.3 the authors state that *’the data was annotated by a group of experts who work on gender studies in political science’*. Until the limitations section, the reader is left to wonder about the IAA and how the final labels were assigned -- these aspects should be mentioned earlier (i.e., in the same section). It would have been good to have even a small sample of the dataset labelled by all the annotators. Does the error analysis reveal any correlations between ‘misclassified’ instances and the person who labelled them?

- On line 201, the authors mention four trigger events, but only three are presented in Table 1.  Considering that on line 907, three trigger points are mentioned, I assume this is the correct number?

- Llama 3 was used for the emotion extraction task – how comes the authors did not include this model too in their analysis?


Suggestions:

- line 217: 1273 -> 1,273
- footnotes should follow (most of the) punctuation marks
- lines 494-496: the ending of the sentence might have to be rewritten
- line 859: stemk -> stem

questions:
Please see the above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730724765167

thread_type:
official_review

<thread_object>
creator: reviewer_i5fu

<content>
summary:
The work  investigates performance of multiple LLMs in the task of detecting sexism in political discourse and proposes a new algorithm for capturing the confidence of the LLM predictions in classification tasks.

strengths:
-	The paper targets an important task. 
-	The paper is clearly written.
-	Extensive experiments were conducted.

weaknesses:
-	There are plenty of papers on detecting misogyny, sexism, etc using LLMs in text. I don’t see political discourse offering specific challenges that requires examining this task in this type of content specifically. Further motivation to this aspect of the work is needed.
-	Line 105: I don’t really see prompt engineering as the reason behind generating good annotations with LLMs, prompt engineering is a tool used to create prompts that can be used to interact with LLMs. The LLMs themselves are the reason/major player in the eventual performance observed in classification for a given task. This confusion continues with line 113, as majority of existing studies focused on investigating the performance of LLMs themselves and not the prompts for NLP tasks. 
-	The dataset has some issues that might effect the outcomes of the work. 
	-	Data annotation process is somewhat unclear. For example, starting line 174, the two labels/concepts mentioned are described as if they are mutually exclusive, while the work was introduced as detecting sexist language in political discourse. I assume there are many cases where the text is political and sexist. Other missing details are the number of annotators, their gender (as I believe in such task, gender of annotator can bring its own biases to the process), how many annotations were collected per tweet?, etc 
	-	The data is really small and targets very few politicians under very specific and small number of trigger events, this of course significantly reduces the studies generalizability for the problem. 
	-	Nothing about the quality of the annotations is mentioned. This is key to establish trust in the study built on top of such data.

questions:
-	Line 163: how was this keywords list created? I suggest also sharing/releasing it with the paper. 
-	The paper builds a new dataset for the task/up to my knowledge, there are several existing datasets, why not use them?
-	Due to how unbalanced the data is, I suggest not reporting Accuracy. This will also help save space and increase table 3’s font size. Also, prompt category names should be the same as those listed under 3.2.1.
-	In Sec 4, I suggest the core observations/outcomes of each experiment to be better highlighted int ext.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732651382483

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of the revised version

comment:
We thank all the reviewers once again for their constructive feedback. We have incorporated most of their suggestions in the revised version. We list a few of the major changes for the benefit of the readers:

- We merged most of the points from the 'Background' section into the 'Introduction' as it contributed to building the narrative for the readers. 
- Revised the section on data annotation, and added further information in the appendix.
- We replaced the previous statistical test (logistic regression) for the confidence estimation with another test (point biserial correlation) as it  
 fit with the purpose of our hypothesis test better. 

Overall, we made a few changes in the document to add more information wherever necessary and delete redundant portions. We also changed the title of the paper, as the current one felt more appropriate for our work.
</content>
</thread_object>

</thread>


Metareview:
metareview:
**Summary:**

The authors evaluate the ability of LLMs to identify sexism in online political discourse. They identified online content referring to 3 female UK MPs and prompted several LLMs to perform either few-shot or zero-shot classification using 4 different prompt designs. They find LLMs struggle to varying degrees with this task. However, confidence estimation experiments indicate that LLMs are very confident about these labeling decisions and have consistently lower confidence for incorrect decisions. 

**Strengths:**

- Expert manual annotation of examples for sexism which could be useful to the research community, though there is still ambiguity around the quality of these annotations.

- Reasonably comprehensive comparison across 5 closed and open LLMs.

- Their experiments with uncertainty quantification provide interesting, solid justification for confidence-based measures to identify model errors in sexism detection.

**Weaknesses:**

- The authors provide mostly thorough background/motivation into the challenges to automatic detection of sexist language, though there have been a number of recent works that are not cited here (e.g. [1, 2]) pointing out potential harms in LLM-as-judge settings.  

- Narrow scope focused on only 3 women involved in UK politics during 2022.

**Comment:**

In Table 1, the "sexist" example seems to be a reference to other people's sexism (an article sexualizing the politician) rather than the direct speaker being sexist, like the example given at the beginning of the paper. Is any distinction made in this work, because it seems like the two cases should be handled differently? 

[1] https://aclanthology.org/2024.emnlp-main.474/

[2] https://arxiv.org/abs/2404.13076