A Diachronic Analysis of Gender-Neutral Language on wiki H ow (2024)

Abstract: As a large how-to website, wikiHow’s mission is to empower every person on the planet to learn how to do anything. An important part of including everyone also linguistically is the use of gender-neutral language. In this short paper, we study in how far articles from wikiHow fulfill this criterion based on manual annotation and automatic classification. In particular, we employ a classifier to analyze how the use of gender-neutral language has developed over time. Our results show that although about 75% of all articles on wikiHow were written in a gender-neutral way from the outset, revisions have a higher tendency to add gender-specific language than to change it to inclusive wording.

## introduction gender-neutral language, also known as genderinclusive language, has its roots in the 1970s, when second-wave feminists criticized the generic use of 'he' and of gendered job titles @xcite . the demand for including women linguistically, by using gender-neutral language, has steadily increased since then. beyond that, gender-neutral language further benefits individuals who identify outside the gender binary or when the gender of the person talked about is unknown @xcite . the online platform wikihow claims to be "the world's leading how-to website". 2 but is it also leading in terms of using gender-neutral language? similar to wikipedia, wikihow articles can be edited publicly and all changes are stored in a revision history. a main difference is that articles are not only written by volunteers, but also by wikihow's own experts, possibly suggesting that editing criteria also include aspects of inclusive language. in this work, we study whether this is 1 http://www.wikihow.com/wikihow:mission , accessed 6 december 2023 2 http://www.wikihow.com/wikihow:about-wikihow , accessed 6 december 2023 the case based on articles written in english, the primary language used on wikihow. many articles, such as how to pack for a holiday, address the reader directly, using the genderneutral pronoun 'you'. however, there are also articles showing that gender-neutral language is not implemented by all editors. for example, the article how to address a congressman uses the term 'congressman' in the title and the gendered phrase 'congressman and congresswoman' throughout the article text. even though this phrase avoids the generic masculine, it is still not gender-neutral as it may not address, for instance, individuals outside the binary. a gender-neutral replacement here would have been the term 'congressperson'. in general, different factors may contribute to the implementation of gender-neutral language in instructional texts. as a first step towards their analysis, this work seeks to answer the following questions: 1) how common are gender-neutral articles in wikihow? 2) how did the ratio change over time? 3) are specific users responsible for corresponding revisions? other challenging tasks, in the landscape of gender and language studies, are sexist language detection @xcite and heteronormative language detection. they identify specific aspects of language that can additionally lead to bias. in the sexist language detection shared task (exist 2021, exist 2022) the goal is to identify hostile, subtle and/or benevolent sexism in english and spanish tweets towards women @xcite . in contrast, the goal of the heteronormativity language detection is to identify heteronormative assumptions in a text. heteronormativity is a "social, political and economic regimen @xcite the only acceptable and normal form to express sexual and affective desires (. . . ) is heterosexuality" @xcite . instructional text have, among other things, been used to analyzing their structure to create instructional text and answer how-to questions @xcite or extract procedural knowledge @xcite . an online-platform that offers a variety of instructional texts on different topics is wikihow. wikihow has served as a source of information for numerous research papers. for example, to detect a users' intent @xcite or to create a summarization tool @xcite . in their paper, @xcite investigated how edits of users can improve texts. if they only improve the instructions' style and correctness, or if they also provide clarifications needed to follow the instructions and achieve the goal. they addressed various types of revision in their paper, such as spelling/grammar, paraphrase, information deletion, and information modification/insertion. however, neither of these types explicitly address gender or gender-neutral language. gender-neutral language is especially important in how-to guides that are addressed to a general audience. ## data and annotation during the beginning of this work, in february 2023, wikihow still offered the export pages service, also referenced by @xcite , for downloading articles and revision histories. we were able to scrape 11, 074, 729 versions of a total of 256, 455 articles using this service. we selected a small subset of these articles to first manually annotate whether gender-neutral language is used. following our original intuition that gender-neutral language may get implemented in articles over time, we specifically searched for article versions in which the phrase 'gender' appears in the comment of a revision. from this set, 129 articles were selected. due to the length of the full articles, we split them into paragraphs for the annotation. one of the authors annotated each paragraph using the labels gendered, gender-neutral and no gen-der, following the definitions provided in a second annotator also annotated 20 articles for reproducibility and quality control. the agreement between the author and the second annotator was very high, with κ = 0.912 @xcite . a total of 2, 247 paragraphs were annotated, with 725 labelled gendered and 1, 235 labelled gender-neutral/no gender. on the article level, we combine the labels as follows: if there is at least one paragraph with the label gendered, the full article is labeled gendered. all the remaining cases are labeled gender-neutral. as a result of this step, 29 annotated articles are labelled gender-neutral, while the remaining 100 are gendered. ## pilot study for classification, different rule-based and supervised variants were compared. since genderneutral language can broadly be defined in terms of specific features, we focus on the following rulebased classifiers: finally, we also experimented with different supervised classifiers, but we did not observe any improvements over the rule-based classifiers. as shown in the unbalanced setting, with 100 gendered articles out of 129 (77.5%), makes it particularly easy to identify the majority class. in in an error analysis, we found that one issue of static list and other rule-based classifiers is that gendered terms can also be used as meta language, which should be classified as gender-neutral. for instance, some articles discuss topics related to transgender or queer issues and what terms can/cannot be used in what contexts: "an example of misgendering would be using she/her pronouns for someone who actually uses they/them, or assuming somebody with long hair is a girl." (from the article how to avoid misgendering). ## analysis we apply the best-performing classifier from our pilot study, static list, to all article revisions collected in the creation of our data ( §3). the following subsections discuss three analyses to answer the questions outlined in section 1. first, we examine the overall distribution of gendered and gender-neutral articles according to their latest version ( §5.1). we then take a look at how this distribution changed over time ( §5.2). finally, we investigate the direction of revisions and check how different editors contributed to it ( §5.3). ## conclusion the objective of this work was to analyze the gender-neutral language of instructional texts. for this, a new dataset of revisions from wikihow articles was created. the annotated gold dataset consists of 129 selected versions of how-to guides, 100 of which are gendered and 29 gender-neutral. a comparison of different classifiers, mostly inspired by previous work, showed that a static word list performed best on our data. a main advantage of static word lists is the option to clearly define which words are considered gendered or gender-neutral, making classifications simple and explainable. in contrast, other classifiers, such as lexical inference may pick up on features associated with biological sex when detecting gen-dered language, which can lead to misidentification of binary and non-binary trans individuals. finally, we classified and analyzed a dataset of over 256, 000 wikihow articles with a total of more than 11 million article versions. our findings discussed in section 5 suggest that, even though most articles start out as gender-neutral, there has been no concentrated effort of editors to change gendered article versions to be gender-neutral. nonetheless, we found several revisions in our annotation study, in which editors implemented gender-neutral language and explicitly mentioned this in the comment of the revision.