Gender Bias beyond Occupational Stereotype
Authors: Ahmed Sabir, Rajesh Sharma


Reviewer paper summaries: In this paper, the authors present a study of gender bias on different datasets. In performing their studies, the authors present a new dataset called GenderLexicon generated with the support of GPT and a framework (unnamed) for contextual bias estimation.

The paper explores the relationship between gender and contextual biases in language models, extending the analysis beyond traditional occupational stereotypes to include action verbs and object nouns. The authors introduce a novel dataset called GenderLexicon and propose a framework that quantifies contextual bias and its related gender bias by using a new scoring mechanism called ClozeGender. This methodology emphasizes the importance of interpreting gender bias in a broader context, including verbs and nouns, and confirms the existence of impactful gender biases beyond occupational stereotypes. They validate their approach using diverse datasets including WinoBias and WinoGender, highlighting that verbs and nouns contribute similarly or even more to gender bias compared to occupations.
Summary of strengths: - Wide experimentation setup;
- Interesting research that could interest some broad or narrow sub-communities in NLP;

- The introduction of the GenderLexicon dataset is novel and broadens the scope of analyzing gender bias by including verbs and nouns along with occupations, allowing for a more comprehensive exploration of gender biases in language models. 

- The authors validate their methodology on four diverse datasets, ensuring the robustness and generalizability of their findings. The extensive evaluations demonstrate the effectiveness of their framework in capturing contextual biases. By focusing on verbs and nouns in addition to occupations, the study addresses a broader range of gender biases, offering a more detailed understanding of how biases manifest in different linguistic contexts, i.e. the findings showing verbs and nouns contribute to gender bias.
Summary of weaknesses: - Some paragraphs seem disconnected from each other;
- Some methodological processes are not clearly defined, which makes research reproducibility difficult

- This work is highly dependent on the quality of the pre-trained word embeddings used. If these embeddings are biased or of poor quality, it could affect the accuracy and reliability of bias detection and measurement. Especially, no matter whether the embeddings are biased or not, the way adapting them to LMs in this work, would probably harm performance on downstream tasks, while no experiments or analysis is conducted on this aspect. The study of gender bias is to expect more accurate and safe LMs, without consideration to this end could be a loss to the analysis part of the paper.

- The process of annotating requires significant manual effort, especially in extracting action verbs and object nouns. Also, his manual intervention can introduce human biases and errors, making the process less scalable and potentially impacting the consistency of results.

- Lacks strong evidence to show how ClozeGender Score improves compared to other gender bias evaluation metrics, especially log likelihood-based metrics. The design of the Score looks natural, which means no significant novelty to this area. The paper compares ClozeGender Score with the metric of WEAT, which is a dataset-specific metric, leading to a potentially unfair comparison.
Comments, suggestions and typos: The paper proposal is exciting because it addresses a relevant research topic. Despite the paper's merits, some improvements must be made. So, I would like to bring light to some issues:

- Throughout the paper, some parts do not seem connected. It is important that, as far as possible, one paragraph is linked to another to avoid breaking the reading flow of the work.

- Throughout the paper, some acronyms are overused, and no explanation of their meaning is provided. A clear example is the "KL" use, a key point to the paper's understanding. All acronyms used in the paper must be presented in its extended form in the first use.

It is not clear from the proposal whether the dataset will be made available for other researchers to use. Providing access to the dataset can enhance the reproducibility and credibility of your research.

- Avoid using contractions in academic writing (e.g., line 115).

The paper needs more organization in the presentation of its content. In some parts, it is a bit confusing to understand on first reading, where it is the author's proposal and where it is something already stated in the literature. This implies hard-to-follow achievements, which do not favour the paper.

Even though the research in NLP is more focused on English content, it is important to make clear to the reader the idiom of the datasets proposed and studied. Developing NLP work in English is common but not necessarily the rule.

When mentioning the proposed dataset in section 1, it would be great to have an example of instances generated to compose the dataset. Explaining only the pattern requested might leave space for confusing interpretations.

-  Multiple references must be done in chronological order. It should be (Zhao et al., 2018; Rudinger et al., 2018; Vig et al., 2020; Kotek et al., 2023) instead of (Zhao et al., 2018; Rudinger et al., 2018; Kotek et al., 2023; Vig et al., 2020)

- Minor issue: There is a typo in the legend figure 7. It should be "ClozeGender" instead of "ClozeGener".

- Mix use of ChatGPT-3.5, ChatGPT-4, GPT3.5, GPT-4
- Lacks references/comparisons on metrics.
Overall assessment scores: 3

3