AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters (2024)

Abstract: Large language models’ (LLMs) abilities are drawn from their pretraining data, and model development begins with data curation. However, decisions around what data is retained or removed during this initial stage are underscrutinized. In our work, we ground web text, which is a popular pretraining data source, to its social and geographic contexts. We create a new dataset of 10.3 million self-descriptions of website creators, and extract information about who they are and where they are from: their topical interests, social roles, and geographic affiliations. Then, we conduct the first study investigating how ten “quality” and English language identification (langID) filters affect webpages that vary along these social dimensions. Our experiments illuminate a range of implicit preferences in data curation: we show that some quality classifiers act like topical domain filters, and langID can overlook English content from some regions of the world. Overall, we hope that our work will encourage a new line of research on pretraining data curation practices and its social implications.

heading: 1 Introduction
text: Large language models (LLMs) are sometimes described to be general-purpose (e.g. Radford et al., 2019), and are increasingly incorporated into realworld applications. However, their behavior can reflect a limited set of human knowledge and perspectives (Johnson et al., 2022; Durmus et al., 2023; Atari et al., 2023). Since the composition of pretraining data has been shown to impact model behavior (Kandpal et al., 2023; Razeghi et al., 2022; Chang et al., 2023; Gonen et al., 2023), documentation of this data facilitates informed and appropriate application of models (Gebru et al., 2021).
In our work, we argue that it is additionally important to examine how data is transformed prior to pretraining, and document the implications of these

heading: About Me
text: transformation steps. LLM pretraining data curation involves many decision points, which may be motivated by performance on popular benchmarks (e.g. Rae et al., 2021), or simply by some notion of text “quality.” There remain many under-examined assumptions within data curation pipelines, which vary subtly across models (Soldaini et al., 2024; Penedo et al., 2023).
We provide a new dataset and framework, AboutMe, for documenting data filtering’s effects on web text grounded in social and geographic contexts. Sociolinguistic analyses in NLP are limited by a lack of large-scale, self-reported sociodemographic information tied to language data (Holstein et al., 2019; Andrus et al., 2021). Though text can be attributed to broad sources, e.g. Wikipedia, the backgrounds of content creators at more granular levels are often unknown. In particular, web crawl data lacks consistent and substantive user metadata. Our study leverages existing structure found in web data. Specifically, some websites include pages delineated to be about the website creator, such as an “about me” page (Figure 1). Thus, we are able
7393
to identify whose language is represented in web scraped text at an unprecedented scale.
From websites’ ABOUT pages, we measure their topical interests, their positioning as individuals or organizations, their self-identified social roles, and their associated geographic locations (§2). We then apply ten “quality” and English ID filters drawn from prior literature on LLM development (§3) onto these websites, show whose pages are removed or retained, and investigate possible reasons for filters’ preferences (§4). Together, our experiments uncover behavioral patterns within and across filters tied to aspects of websites’ provenance. We find that model-based “quality” filters’ implicit preferences for certain topical domains lead to text specific to different roles and occupations being removed at varying rates. In addition, English content associated with non-anglophone regions of the world can be removed due to filtering approaches that assume pages are monolingual.
We release our dataset, reproduced filters, and other resources to facilitate future work:
Code github.com/lucy3/whos_filtered Dataset huggingface.co/datasets/
allenai/aboutme

heading: 2 Extracting Social Dimensions from ABOUT Pages
text: Sociolinguists conceptualize language as a performance of one’s social identity, or membership in a social group (Nguyen et al., 2016). Websites’ ABOUT pages capture aspects of their creators’ social identities that they deem salient and significant enough to mention in a summary (Figure 1). Thus, these self-descriptions can help delineate meaningful differences in language varieties and use. We extract social aspects that are present across large sets of pages using automated methods, some of which we contribute as novel approaches. We do not examine attributes such as race or gender, as these are less commonly explicitly stated on ABOUT pages and may raise the risk of mismeasurement (§8).

heading: 2.1 Data preprocessing
text: AboutMe is derived from twenty four public snapshots of Common Crawl collected between 2020–05 and 2023–06. We extract text using CCNet (Wenzek et al., 2020) and deduplicate URLs across all snapshots. Our study focuses on data curation of English LLMs, and our pipeline for identifying social aspects of websites uses methods that work best for English. Thus, we limit our study to CCNet’s outputted webpages that have a fastText English ID score > 0.5 (Joulin et al., 2016b,a).
From this Common Crawl data, we identify websites that include an ABOUT page, or URL paths containing about, about-me, about-us, or bio (Appendix A). We then pair each ABOUT page with a random page on the same website. AboutMe thus contains both information about the creator/s of a website and a sample of their textual content (Table 1). Though we use this dataset to study LLM data curation practices, text linked to their creators’ self-descriptions can also facilitate research on selfpresentation (Sun et al., 2023; Pathak et al., 2021) and language variation (Nguyen et al., 2016).

heading: 2.2 Topical interests
text: First, we treat ABOUT pages as summaries of website creators’ interests and topical focus. Following past work on the unsupervised discovery of domains (Gururangan et al., 2023), we embed ABOUT pages using unigram counts and tf-idf (Manning et al., 2008) and cluster them with balanced kmeans. We set k = 50 and surface a wide range
of topical clusters in our data, including design, finance, food, religion, and travel (Figure 2, Appendix C). Thus, this clustering step provides an initial broad overview of who and what is in AboutMe.

heading: 2.3 Individuals vs. organizations
text: Website creators can range from casual bloggers to larger corporations. We deem webpages with urls that contain about-me or bio as individuals, while those that are labeled as about-us are organizations. However, some pages are ambiguously labeled with about, and so we classify these into individuals or organizations by training a binary random forest classifier on labeled pages. Classifier inputs include several count features that are agnostic to pages’ topical content: the proportion of words on an ABOUT page in common pronoun series (he, she, they, we, I), the proportion of words that are tagged as a PERSON by spaCy named-entity recognition, and the number of unique PERSON first tokens. Our classifier achieves an average macro F1 of 89.2, via 5-fold cross validation on 10k examples per class. Hyperparameter choices, classifier confidence, and other implementation details are in Appendix D.1. By applying this classifier, we find that three-fourths of hostnames are organizations rather than individuals (Table 1).

heading: 2.4 Social roles
text: Among individuals, we extract their self-identified social roles from their ABOUT pages. The salience of a social role or occupation in a setting impacts language. Roles not only shift text’s topical focus, but also facilitate the use of situation-specific language styles and registers (Agha, 2005).
String-matching can be imprecise due to polysemy and mentions of other people on ABOUT pages (e.g. a customer). Thus, our role extraction approach targets explicit expressions of selfidentification (e.g. I am a designer, entrepreneur, and mother). We hand-label a sample of 1K ABOUT page sentences spanning a diverse set of potential roles,1 and treat role extraction as a binary, sentence-level token classification task. Our full criteria for role annotation can be found in Appendix F.1, and we achieved high agreement (Cohen’s κ = 0.836).
We finetune ROBERTA-base on our labeled data, as it provides a scalable yet flexible approach
1https://en.wiktionary.org/w/index.php?title= Category:en:People
Occupation family Count Examples of extracted roles
Arts, Design, Entertainment, Sports, & Media 1.1M artist, director, designer, writer, photographer, musician, player Production 620K designer, engineer, maker, builder, operator, mechanic Community & Social Service 452K therapist, educator, advisor, pastor, activist, social worker Computer & Mathematical 365K engineer, developer, scientist, strategist, programmer Educational Instruction & Library 308K teacher, professor, lecturer, curator, tutor, graduate student
Table 2: Five most common occupation families in AboutMe, by website count, with example social roles. Additional examples can be found in Appendix F.2-F.3.
for learning a variety of self-identification patterns. Before finetuning, we continue pretraining ROBERTA on individuals’ ABOUT pages, improving in-domain performance (Gururangan et al., 2020). Hyperparameters and model selection details can be found in Appendix F.2. Our best model achieves a F1 of 0.898 when evaluated at the wordlevel on a held-out test set.
With this approach, we are able to identify social roles on 77.7% of all individuals’ ABOUT pages (Table 1), and pages that have any roles contain 5.5 on average (SD = 9.9). When possible, we group terms into occupations based on a taxonomy created by the Occupational Information Network, or O*NET (Peterson et al., 2001), e.g. the roles attorney and lawyer are in the occupation of Lawyer in the Legal occupation family (Table 2, Appendix F.3). For our filtering rate analysis (§4), we include 780 social roles that occur at least 1K times in AboutMe.

heading: 2.5 Geography
text: Models’ emphasis on English already restricts their ability to capture perspectives from people who write in other languages, especially populations outside of Western, anglophone countries (Blasi et al., 2022; Durmus et al., 2023). Still, English is commonly chosen for intercultural communication and sometimes characterized as a world language or lingua franca (Seidlhofer, 2005). Thus, our web-derived dataset includes a range of geographic contexts in which English is used.
We geoparse locations on ABOUT pages using Mordecai3, which tags named locations, retrieves candidate matches from a GeoNames index, and disambiguates them using textual context (Halterman, 2023). For example, I’m from Alexandria, Virginia would be geoparsed to a location in the United States instead of Egypt. Mordecai3 is free and uses a local index, and so it can be scaled
up to millions of webpages. With this approach, we are able to tag 63.2% of websites in AboutMe with at least one country. This coverage exceeds the 10.38% obtained by using country-specific toplevel domains, e.g. .uk for the United Kingdom (Cook and Brinton, 2017).
Locations mentioned on a page are often associated in some way with the creator/s of a website, but the strength of this association can vary. For example, a company may have been founded in one country, but ships products to another. Through manual annotation of locations in 200 ABOUT pages, we find that 79.46% of websites with geoparsed countries originate from or reside in the most frequently referenced country on their ABOUT pages (evaluation details in Appendix G.1). Thus, we label each website with its most frequent country, yielding an overall website-level labeling accuracy of 91.0%. Due to the current global digital divide and our focus on English, the majority of websites in AboutMe are labeled with the United States and United Kingdom, with a long tail of other countries (Figure 3). For analysis, we group countries into 5 continental regions and 15 subregions delineated by the United Nations (Appendix G.2).

heading: 2.6 Summary
text: Overall, the websites in AboutMe cover a variety of topical interests, though a large proportion are associated with locations in the United States. A majority of websites are by organizations rather than individuals, and among individuals, most websites are created by people with creative and mediarelated occupations. Finally, though our methods for characterizing ABOUT pages achieve good performance, they still have limitations and mismeasurement risks, which we discuss in §7 and §8.
Filter Examples of prior use Removal strategy ★WIKIWEBBOOKS, or Wikipedia, OpenWebText, & Books3 classifier GPT-3 (Brown et al., 2020) Sampling based on scores ★OPENWEB, or Reddit outlinks classifier the Pile (Gao et al., 2020) Sampling based on scores ★WIKIREFS, or Wikipedia references classifier LLaMA (Touvron et al., 2023a) & RedPajama (Computer, 2023)
Cutoff: 0.25 (RedPajama), binary (LLaMA)★WIKI, or Wikipedia
classifier Specified in reference mixes by Xie et al. (2023), PaLM (Chowdhery et al., 2023), and GPT-3 (Brown et al., 2020) Sampling based on scores ★WIKIppl, or Wikipedia perplexity CCNet (Wenzek et al., 2020) Percentile cutoffs: 33.3% or 66.7% ★GOPHER length, wordlist, repetition, & symbol rules
Gopher (Rae et al., 2021), Chinchilla (Hoffmann et al., 2022), & RefinedWeb (Penedo et al., 2023) Specific cutoffs for each rule
∗fastText classifier CCNet (Wenzek et al., 2020), LLaMA (Touvron et al., 2023a), RefinedWeb (Penedo et al., 2023) Cutoffs: 0.50 (CCNet, LLaMA), 0.65 (RefinedWeb) ∗CLD2 classifier The Pile (Gao et al., 2020) Cutoff: 0.50∗CLD3 classifier multilingual C4 (Xue et al., 2021) Cutoff: 0.70 ∗langdetect classifier C4 (Dodge et al., 2021; Raffel et al., 2023) Cutoff: 0.99
Table 3: Quality filters (★) and langID systems (∗) investigated in our study.

heading: 3 Pretraining Data Filters
text: Raw data is transformed into pretraining data for LLMs through a variety of steps (Penedo et al., 2023; Soldaini et al., 2024, inter alia), including deduplication (Lee et al., 2022), decontamination (e.g. Touvron et al., 2023b), and explicit content filtering (e.g. OpenAI, 2023). We focus on analyzing the effects of “quality” filtering and English langID. The former is motivated by the subjectivity of how quality should be defined, and the latter by ongoing uncertainty around whether langID is robust to a wide range of language varieties (Seargeant and Tagg, 2011; Caswell et al., 2020).
Since web text contains noisy content (Eisenstein, 2013), removing or downsampling “low quality” text is common in LLM development (Table 3). However, mismatch between filtering outcomes and downstream objectives can lead to performance degradation on some tasks (Gao, 2021; Longpre et al., 2023), or disfavor content written by minoritized populations (Gururangan et al., 2022). LangID is also a common step in data curation pipelines (Table 3). It can be used at the documentlevel as an initial filter for language-specific (e.g. English-only) models, or to measure and adjust the composition of pretraining data for multilingual models (Xue et al., 2021). However, popular
langID systems are imperfect, for reasons such as training and application domain mismatch and confusion between similar languages (Kreutzer et al., 2022; Caswell et al., 2020).
We critically examine ten document-level quality and English filters that are sufficiently documented in prior work (Table 3). Appendix B includes additional details on the reproduction of each filter. Descriptions of pretraining data curation are sometimes too vague or non-existent to allow for exact replication (OpenAI, 2023), but multiple recent and prominent LLMs still allude to the use of model- and heuristic-based data filters (Touvron et al., 2023a; Gemini Team et al., 2023; Chowdhery et al., 2023).
Model-based quality. We experiment with quality filters that score text based on their similarity to some chosen “high quality” reference corpora. We name these filters based on the reference corpora used to train them: WIKIWEBBOOKS, OPENWEB, WIKI, and WIKIREFS (Table 3). We use Gururangan et al. (2022)’s replication of GPT-3’s binary logistic regression quality classifier and only vary the positive “high quality” class. The negative class is a fixed set of tokens from the September 2019 dump of Common Crawl, and each class contains approximately 300M tokens. We also compare WIKI to a perplexity-based text scorer, WIKIppl, which uses a 5-gram Kneser-Ney language model trained on Wikipedia instead of a classifier (Wenzek et al., 2020; Laurençon et al., 2022; Muennighoff et al., 2023; Marion et al., 2023).
Heuristic-based quality. Another quality filtering approach for web text applies rule-based heuristics (Raffel et al., 2023; Rae et al., 2021). We examine 19 document-level heuristics and thresholds from Gopher (Raffel et al., 2023). These heuristics remove documents that do not meet thresholds pertaining to document and word length, textual repetition, and frequencies of symbols and common English words (Appendix B).
English langID. Our data is pre-filtered to documents that fastText langID scores as likely English (Joulin et al., 2016b,a). We also investigate whether the range of scores we observe with fastText generalize to other model-based measurements of English used in LLM development (Table 3). These langID systems include Compact Language Detector 2 (CLD2) (Sites, 2013), CLD3 (Salcianu et al., 2020), and langdetect (Shuyo, 2014).

heading: 4 Whose websites are filtered?
text: In this section, we overview the effects of data filters on sampled webpages grouped by social dimensions identified from their ABOUT pages. Broadly, we examine the degree of consensus among filters when scoring pages, and identify themes that characterize their behavior. Within some dimensions, we also investigate whether filtering rates reflect systemic differences in power and status among social groups (Blank, 2013; Davis, 2018).
Past LLMs have chosen a range of score cutoffs and sampling mechanisms to reduce undesirable text (Table 3). With this variation in mind, we examine the outcome of model-based filters through the lens of two contrasting scenarios. First, whose pages are least affected, or retained, if we were to keep only the documents within a top percentile of scores? Second, whose pages are most affected, or removed, if we were to filter those at a very bottom percentile? We select top and bottom cutoff percentiles of 10% and 90%, though for CLD2 and langdetect, a large number of score ties meant that that cutoffs for both scenarios were 5.2% and 8.7%, respectively. For rule-based filters, we use cutoffs specified by Rae et al. (2021). All cutoffs are listed in Appendix B.

heading: 4.1 Topical interests
text: Similarities in how data filters score topical clusters cut across quality and English filters (Table 4, Table 5). Pairwise correlations of topics’ average English scores across all four langID systems have high consensus (mean rs = 0.874, SD = 0.038, all p < 0.001). Surprisingly, Wikipedia perplexity also behaves like fastText langID (rs = 0.860, p < 0.001). We qualitatively examine 20 random pages from highly filtered clusters, e.g. fashion, women and online, store. We find that pages with lower English scores list product names or specifications of individual products, and their original content may have been highly visual.2 Indeed, further down the list of commonly highly filtered topical interests are clusters related to photography and art (Appendix C.2). Thus, though text-based LLMs may intend to be comprehensive in knowledge, they exclude information that is primarily communicated via other forms of media.
Differences among topical preferences show that the method of filtering and the choice of reference
2Indeed, manual inspection of current versions of these websites, when available, supports this claim.
A few topical interests that recur throughout the table are highlighted for clarity. See Appendix C.2 for an extended and more detailed version of this table.
corpora can influence what “quality” entails. Despite both being trained on Wikipedia, a perplexitybased filter behaves differently from a linear classifier (rs = 0.382, p < 0.01). In addition, a quality classifier’s behavior reflects the composition of its reference corpora. For example, classifiers trained to prefer web content outlinked from Reddit or Wikipedia, including WIKIWEBBOOKS, OPENWEB, and WIKIREFS, highly score news and media websites. In contrast, WIKI and WIKIWEBBOOKS tend to prefer topics well-represented on Wikipedia, such as entertainment and science (Mesgari et al., 2015). Thus, these “quality” filters may optimize for topical domain fit.

heading: 4.2 Individuals vs. organizations
text: Research has suggested that “non-standard,” colloquial language can be considered less desirable (Blodgett et al., 2020; Eisenstein, 2013). So, we hypothesize that organizations’ language may be considered higher quality and more “English” than that of individuals, as organizations may have more editorial resources to professionally create their
content (Wagner, 2002).
Surprisingly, we find that across all quality and English langID filters, web content created by individuals is widely preferred (Appendix D.2). For example, GOPHER removes 25.9% of webpages by individuals, and 28.3% of those by organizations. One reason for this pattern is that topics that receive overall low scores by data filters are dominated by organizations, e.g. businesses in the clusters products, quality and online, store. However, even when topics are fixed, organizations are still more likely to be removed by nearly all filters, with only WIKIREFS as an exception (Appendix D.2). Webpages by organizations tend to be shorter than those by individuals across and within topics, and their webpages include more non-alphabetic “words”, more repetition, and fewer words from GOPHER’s required list (all p < 0.001, Mann-Whitney U -test).3
3GOPHER’s required wordlist includes the, be, to, of, and, that, have, with. See Appendix B for details.
(+). For interpretation clarity, roles are highlighted if they belong to four frequently recurring O*NET occupation
families. See Appendix F.4 for an extended and more detailed version of this table.

heading: 4.3 Social roles
text: Filters’ social role preferences mirror those for topical interests (Table 6), such as programming occupations and software, data being preferred by OPENWEB, and correspondent matching the often “high quality” topic of news, media. We measure the relationship between model-based filter scores and two metrics that reflect occupations’ societal status: their O*NET salary estimate and Hughes et al. (2022)’s survey-based ratings of prestige. We find small yet significant relationships between occupational prestige and model-based quality scores (p < 0.001, Appendix F.4). That is, pages linked to lower prestige occupations are filtered more by WIKIWEBBOOKS, OPENWEB, WIKIREFS, WIKI, and WIKIppl. The degree to which pages’ self-identified roles are expressed through their text affects filtering as well (Figure 4). Within each role’s collection of webpages, we calculate the proportion of each page that contains vocabulary specific to that role. Following past work (Zhang et al., 2017; Lucy et al., 2023), we identify role-specific vocabulary using a metric of association between word types and roles, where their normalized pointwise mutual information (NPMI) score is greater than 0.1. We compare how filters score all pages within a role, and how they score a subset of the role’s pages that contain more role-specific words than average. We find that roles that are generally favored by a filter tend to be favored even more when their pages contain more role-specific words, in contrast to roles that are scored lowest by a filter, which do not benefit
from role-specific word use or are penalized further. Exceptions to this pattern are CLD2 and langdetect, two langID filters that score the vast majority (>90%) of pages similarly. These findings suggest that caution may be needed when using pretrained LLMs out-of-the-box for tasks and applications that involve language specific to some domains.

heading: 4.4 Geography
text: One striking commonality among several data filters is that they tend to assign low scores to webpages from Asia (Figure 5). For example, webpages are 2.4 times more likely to be removed by CLD2 if they are associated with Eastern Asia than Northern Europe. Eastern Asia is the most topically skewed subregion, as 29.2% of its websites are in the lowly-scored quality, equipment topic.
However, geographic filtering patterns are not only explained by topical differences. As expected, most English filters prefer subregions with “core anglophone” countries: Northern America (Canada and US), Northern Europe (UK), and Australia & New Zealand (Figure 5). Subregions with lower English document-level scores contain more non-English paragraphs across all langID systems (mean r = -0.791, SD = 0.084, all p < 0.05). By examining these “non-English” paragraphs, we observe two reasons for why a page may not be “English” enough. First, langID can mislabel English text (Caswell et al., 2020), such as content containing names of products, people, and non-anglophone locations. Second, some web pages are indeed multilingual, either code-switching or including multiple translations of the same content. LangID is usually applied at the document-level during data curation, and some systems may assume monolingual inputs (Zhang et al., 2018). Non-English paragraphs in AboutMe reflect their geography, e.g. Chinese in Eastern Asia, Spanish in Latin America, and Polish in Eastern Europe. Thus, simply choosing to communicate in English is not necessarily grounds for inclusion, and how English is situated within webpages matters.
In addition, we examine how filtering of geographic locations may relate to their relative global status. Past work has suggested that some NLP models may favor wealthier countries (Zhou et al., 2022). In our case, we do not find a significant relationship between a country’s filter scores and their gross domestic product (Appendix G.3).

heading: 4.5 Regression analysis
text: Finally, we investigate the relative importance of the social dimensions highlighted in previous sections for model-based filters. That is, what matters more: who you are, or where you are from?
We run several ordinary least squares regressions with different filters’ scores as dependent variables, and topical interests, continental regions, individual/organization status, core anglophone status, and pages’ character length as independent variables. All independent variables, except for length, are binary or categorical. We z-score standardize each regression’s dependent variable so that coefficients are similarly interpretable across them. We find that length has a positive effect (p < 0.001) on all model-based filter scores except for WIKIppl,
and many topical variables have stronger effects on filter scores than other variable types, especially among quality classifiers (Figure 6, Appendix H). Though earlier we noted that some subregions’ filtering scores may be due to topical skew (§4.4), even when controlling for topic, Asia still has the most negative coefficients relative to other continental regions for all langID filters.

heading: 4.6 Summary
text: We find shared patterns in how quality and English filters score websites delineated by social aspects such as social roles and geography. Differences in how quality filters behave depend on both how they are implemented and their choice of “high quality” reference corpora (§4.1). This latter factor results in notions of “quality” being associated with certain topical domains e.g. news and media, and these topical preferences then lead to filters privileging content specific to some social roles over others’ (§4.3). Finally, common langID classifiers can overlook English content in non-anglophone regions of the world, especially Asia, even when controlling for other variables such as webpage length and topic (§4.5).

heading: 5 Related Work
text: Our measurements of self-descriptions are most related to prior work studying online selfpresentation. Online biographies are a particularly rich source of social identity markers. For example, Pathak et al. (2021) extracts personal identifiers, e.g. farm wife or umass amherst ’20, from Twitter profile bios, and show that on aggregate, identifiers in these bios align with users’ sociodemographic backgrounds. Others have extracted identifiers by splitting short form content by delimiters, e.g. 22yo | she/they → {22yo, she/they} (Yoder et al., 2020; Pathak et al., 2021), or matching syntactic patterns, e.g. person is X (De-Arteaga et al., 2019; Madani et al., 2023). As one of the self-description analysis tools we employ, we contribute a novel and more flexible RoBERTa-based approach in §2.4 for extracting social role identifiers.
Though a nascent research area, analyses of pretraining data curation decisions have also been the focus of other recent literature (Longpre et al., 2023; Soldaini et al., 2024). For example, Gao (2021) showed that discarding too much pretraining data using the Pile’s quality filter can lead to worse downstream task performance. Our work is
closest in spirit to Gururangan et al. (2022), who use a dataset of high school newspapers to show that text from wealthier, more educated, and urban areas are more likely to be considered high quality by GPT-3’s model-based quality filter. Similarly, concurrent work by Hong et al. (2024) found that image-text CLIP-filtering for visual language models excludes data from LGBTQ+ people, older women, and younger men at higher rates. We also critically examine social aspects of quality filtering at scale, but across a range of text filters.

heading: 6 Conclusion
text: In our work, we examine how ten “quality” and English langID filters used during LLM development affect web text created by a range of individuals and organizations with different topical interests, social roles, and geographic locations. To obtain this information, we use a new dataset of webpages that contain website creators’ self-described social identities. Overall, our framework allows for model developers and practitioners to better understand whether and how their choice of filtering approach may affect the resulting composition of web data in unintended ways. Though some practices may seem tried and true for building powerful LLMs, we encourage future work to continue investigating, documenting, and mitigating their caveats and tradeoffs.

heading: 7 Limitations
text: Algorithmic measurements of websites allow our investigations to scale to millions of webpages. Still, we acknowledge that our dataset and analysis methods can also uphold language norms and standards that may disproportionately affect some social groups over others. For example, AboutMe consists of documents that meet a Fasttext langID English score threshold of 0.5, as the algorithmic tools we use for later analyses are created for English. There are likely some false negatives we excluded from analysis, as some English content may not meet this threshold. As another example, named entity recognition during geoparsing may rely on locations being stated using standard capitalization norms in text. Our study also focuses only on English, due to a current gap in multilingual tools for large-scale data documentation (Joshi et al., 2020). We hope that future work continues to improve these content analysis pipelines, especially for long-tail or minoritized language phenomena.
We study the effects of filters in isolation, but acknowledge that in practice, data curation steps are layered and combined. The exact preprocessing of text before filters are applied may impact outcomes; for example, some langID systems can be applied to web data prior to HTML removal (Gao et al., 2020). Unfortunately, commercially prominent LLMs often lack detailed documentation necessary for investigations at this level of specificity. Still, we encourage future work to investigate implications of layered LLM data curation practices.

heading: 8 Ethical Considerations
text: This work received IRB exemption, and includes several ethical considerations.
Measurement error. Our analysis approach leans more towards extraction of stated information and less towards inference of additional information. That is, we aim to minimize the extent to which we impose implicit labels on people. Still, we risk cases where websites are misidentified due to retrieval or identification error. Measurements of social identity from ABOUT pages are affected by reporting bias, where a lack of self-provided information can lead to pages being excluded from relevant analyses. We encourage future work to revisit these issues, while adhering to privacy-related principles in mind.
Pronouns. Exclusivity and misrepresentation harms towards non-binary people have been gaining attention in the NLP community (Dev et al., 2021; Cao and Daumé III, 2020). We recognize that in the process of measuring different aspects of who is filtered, websites by non-binary individuals are likely mishandled by the algorithmic approaches we use. That is, our classifier discerning individuals and organizations relies on common pronoun series as input features, but some non-binary people may use neopronouns, e.g. xe/xem/xyr (Lauscher et al., 2022; Ovalle et al., 2023). In addition, the models we leverage, such as spaCy and RoBERTa, may mishandle text containing neopronouns. Neopronouns, though rare, do appear in AboutMe; we surface approximately 21 websites whose ABOUT pages’ most frequent pronoun series are neopronouns (Appendix E).
Intended use of dataset. Though the data we analyze is provided by Common Crawl, a source of publicly open web data, care still needs to be taken when handling this data. Future uses of this data
should avoid incorporating personally identifiable information into generative models, report only aggregated results, and paraphrase quoted examples to protect the privacy of individuals (Bruckman, 2002).
Other considerations. Throughout this paper, we describe the effects of exclusion of data from pretraining as potentially perpetuating erasure or decreasing downstream model performance on relevant tasks. However, removal from pretraining data is not always a negative outcome. For example, in some cases, content creators may prefer that their content is not incorporated into training LLMs due to copyright violation and/or lack of consent.

heading: 9 Acknowledgements
text: We thank Nicholas Tomlin, Naitian Zhou, and Kyle Lo for helpful conversations and feedback. We also thank our anonymous reviewers for their thoughtful reviews. This work was supported in part by the National Science Foundation (IIS-1942591).

heading: A Data preprocessing
text: Our dataset, AboutMe, consists of ABOUT pages identified using webpage URLs (§2). Some webpages have multiple pages with URLs involving a target keyword (one of about, about-me, aboutus, or bio). We retrieve ABOUT pages that end in /keyword/ or keyword.*, such as a URL ending in about.html. If there is only one of these candidates, we map the hostname to that one. If there are more than one, then we do not include that hostname in AboutMe, to avoid ambiguity around which page is actually about the main website creator. If a webpage has both https and http versions in Common Crawl, we take the https version.
Aside from cases where tokenizers are built into models or systems we use to analyze text, e.g. ROBERTA or Mordecai3, we use Microsoft’s Bling Fire tokenizer.4

heading: B Data filters

heading: B.1 Filter reproduction
text: All model-based quality filters, except for WIKIppl, use the same implementation and parameter choices as the reproduction of GPT-3’s quality filter by Gururangan et al. (2022).
WikiWebBooks. Both positive and negative examples for this classifier are the same as Gururangan et al. (2022). Their positive class consists of similar sized samples from Wikipedia, OpenWebText, and Books3. We reuse the same set of negative examples for other quality classifiers that share the same model architecture: OPENWEB, WIKIREFS, and WIKI.
OpenWeb. The original version of WebText was introduced in the GPT-2 paper, which described it as “all outbound links from Reddit, a social media platform, which received at least 3 karma” (Radford et al., 2019). We use an open and updated version of this dataset constructed by the Pile, called OpenWebText2 (Gao et al., 2020). The Pile uses this version to filter their version of Common Crawl. We sample documents from one shard of OpenWebText2 until we meet a 300M token ceiling to create the positive class for this classifier.
WikiRefs. We sample up to 300M tokens worth of webpages referenced by English Wikipedia to construct the positive class for this filter. We use text previously extracted by Barham et al. (2023).
4https://github.com/microsoft/BlingFire
Wiki. We use text extracted from a dump of Wikipedia from March 20th, 2023.
Wikippl. This perplexity-based KenLM filter trained on English Wikipedia is provided by CCNet, and its download link is specified in CCNet’s Makefile.5
Gopher. We use Dolma’s reproduction of Gopher’s document-level rules for web text quality,6 though we change median word length to mean word length to match the rule’s description in the original Gopher paper (Rae et al., 2021). Table 7 overviews what percentages of webpages in AboutMe are removed by each rule or set of rules.7 Overall, larger proportions of pages do not pass document length and repetition heuristics. Rules include the following, indicated by a shortened name for ease of reference:
• doclen: page length is between 50 and 100,000 words
• wordlen: mean word length is within 3 to 10 characters
• symbol: symbol-to-word ratio is less than 0.1, where symbols are either the hash symbol or ellipsis
• bullet: less than 90% of lines start with a bullet point
• ellipsis: less than 30% of lines end with an ellipsis
• alpha: more than 80% of words in a document contain at least one alphabetic character
• stopword: page contains at least two of the following English words: the, be, to, of, and, that, have, with
5https://github.com/facebookresearch/cc_net/ blob/main/Makefile
6https://github.com/allenai/dolma 7Note that a single webpage can be affected by multiple
rules.
• repetition: no content that exceeds several thresholds related to duplicated content: fraction of characters in most common bigrams (0.20), trigrams (0.18), or 4-grams (0.16), fraction of characters in duplicate 5-grams (0.15), 6-grams (0.14), 7-grams (0.13), 8- grams (0.12), 9-grams (0.11), 10-grams (0.10), fraction of duplicate lines (0.30), and fraction of characters in duplicate lines (0.20).
LangID. We build off of Dolma’s toolkit to apply all langID filters to text (Soldaini et al., 2024). Dolma’s existing functionality outputs English scores for CLD3, CLD2, and fasttext, and we implement analogous functions for applying langdetect. We also calculate paragraph- and sentencelevel language scores for any language. For this, we follow Dolma’s definition of a paragraph (character sequences separated by new lines) and sentence (Bling Fire’s sentence tokenizer).

heading: B.2 Score cutoffs
text: As described in the main text in §4, to investigate webpages that are most or least favored by a filter, we use two cutoffs: a more strict scenario that removes all but the top 10% of scores, and a more flexible one that removes only the bottom 10%. However, two filters, CLD2 and langdetect, contain many score ties, so we instead use the same numeric cutoff for both scenarios, and this cutoff corresponds to the bottom 5.2% and 8.7% percentiles of scores, respectively. Table 8 lists the numeric cutoffs that we used to obtain ↑ retained and ↓ removed results in the main paper (e.g. Table 6, Table 5, Figure 5). We observe that regressionbased classifiers tend to label most Common Crawl webpages with low scores. In addition, among langID classifiers, fastText has the most graded and gradual score distribution, while other langID systems tend to mostly give very high or very low
English scores.

heading: C Topical interests

heading: C.1 Clusters
text: For clustering, we use the same parameter choices as Gururangan et al. (2023).8 We chose k = 50 as the number of clusters, because it offers a level of granularity that yields distinctive and interpretable topical areas. Since this version of k-means is balanced, clusters are encouraged to be similar in size. Figure 7 lists all 50 clusters and their frequency.

heading: C.2 Additional filtering results
text: Table 9 shows the ten most and least Gopherfiltered topics, with a breakdown by rule. We find
8https://github.com/kernelmachine/cbtm
that the top three rules that affect pages within topics are similar; webpages from nearly all topics are highly filtered due to document length being too short. Table 10 is an extended version of Table 5, listing top ten topics instead of the top five.
D Individual and organizations

heading: D.1 Classifier details
text: We separate out websites created by individuals and those by organizations using a random forest classifier. This classifier is trained on 10k randomly sampled about me/bio pages and 10k about us pages, and used to disambiguate about pages. It incorporates the following features:
• Proportion of words that are in each pronoun series: first person singular (I), first person plural (we), third person feminine (she), third person masculine (he), and third person gender-neutral/plural (they).
• Number of PERSON entities, normalized by the word length of the page
• Number of unique PERSON first tokens
To obtain named PERSON entities, we use spaCy’s en_core_web_trf model. We set hyperparameters for our random forest classifier by selecting the best model based on its F1 score, cross validating over 5-folds, and performing randomized search over the following scikit-learn hyperpameters:
• n_estimators: 50, 100, 150, 200, 250, 300 • criterion: entropy, gini • max_depth: None, 10, 50, 70, 100 • min_samples_split: 2, 5, 10, 20 • min_samples_leaf: 1, 2, 4.
Our best model with a F1 score of 0.892 had the following hyperparameters: n_estimators (200),
min_samples_split (20), min_samples_leaf (2), max_depth (70), criterion (gini). Our resulting model tends to be highly confident based on its distribution of class probability scores (Figure 8). In other words, there are few websites that are around the border of what our model considers to be an organization or individual. Qualitatively, an example type of a website that is more ambiguous along the individual vs. organization dimension are businesses whose ABOUT pages tend to focus on the background of their current leader or founder.

heading: D.2 Additional filtering results
text: Table 11 shows filtering rates for individuals versus organizations for the two cutoff scenarios motivated in §4. On average, individuals have higher model-based scores than organizations for every filter, and so when the very top percentile of pages are retained, individuals are retained at higher rates. With regards to Gopher heuristics, 28.3% of organizations and 25.9% of individuals are removed, and the most prominent reason is again document
length (Table 12). Across most filters, individuals within each topic have, on average, higher scores than organizations in the same topic (Table 13).

heading: E Neopronouns
text: Our individual versus organization classifier uses pronoun counts as input features. During the process of gathering these pronoun features, we also examined possible ways to quantify or extract neopronouns from AboutMe. We began with an initial list of pronoun series that includes common neopronouns.9 Some of these pronoun series, such as it/it/its and kit/kit/kits lead to a high number of false positives with exact string matching. In addition, we were not able to disambiguate cases where they/them/theirs is used as a plural pronoun instead of a singular one.
For other pronoun series, we identify potential pages whose subject uses neopronouns by finding ABOUT pages that include at least two unique pronouns from a neopronoun series, and that neopronoun series’ frequency exceeds the frequency of more common pronouns. We manually inspected a sample of pages for each neopronoun series extracted with this approach, and estimate that only∼21 of 10.3M ABOUT pages contain uses of neopronoun terms as pronouns. The most frequent neopronoun series is xe/xem/xyr, with 8 extracted occurrences. Overall, the counts we obtained were too low for inclusion in our study. They are also likely an undercount, as we were only able to verify for precision rather than recall. We encourage future work to consider safe and inclusive studies of pronoun use in self-descriptions.

heading: F Social roles

heading: F.1 Annotation
text: We begin by annotating sentences that possibly contain terms referring to people. We explored two possible options for obtaining a seed list of terms: English Wiktionary’s Category:en:People, and WordNet hyponyms of person. We found that the latter is imprecise (e.g. WordNet lists have as a person due to the phrase haves and have-nots) and outdated. So, we used English Wiktionary’s list as a starting point for capturing a wide and up-todate range of social roles (e.g. influencer). After removing terms that are overly long (4+-grams),10 we string-matched for 10,676 Wiktionary terms on individuals’ ABOUT pages. To avoid overfitting to popular roles, we reservoir sample for one
9https://github.com/witch-house/pronoun.is/ blob/master/resources/pronouns.tab
10These tend to be sayings such as life of the party or big fish in a small pond.
ABOUT page per term, and then sampled 1000 random examples from that pool for annotation. We annotate head tokens of roles in the context of a single sentence, with seed terms pre-highlighted for annotators to verify, add to, or remove. We divide examples for annotation among the authors of this paper, following instructions shown in Figure 11.
Across all 1k annotated sentences, 541 contain at least positively labeled one social role in them. Overall, our annotators marked 1284 unique spans as roles. Thirty-five sentences were doubly annotated. Our annotators had good sentence-level agreement (Cohen κ = 0.836), and only differed on 4 of these sentences total.

heading: F.2 Token classification
text: For finetuning ROBERTA, we grid-search through several learning rate options (1e-5, 2e-5, and 3e5), experiment with varying levels of continued masked-language-modeling pretraining, and use a train-dev-test split of 600/200/200 labeled examples (Table 15). For other parameters, we use the same choices as Gururangan et al. (2020).
Our labeled spans are whole words, but ROBERTA sometimes labels parts of words. When we run inference on all individuals’ ABOUT pages, we find that it nearly always tags all wordpieces in a positive span correctly. Still, 3.3% of tagged words are partially tagged, e.g. play-mate, trend-set-ter, mom-my. From manual inspection of these cases, it seems like partially tagged words are usually social roles. Thus, we evaluate at the word-level, and count words as social roles if any of its wordpieces is tagged as one.
Table 16 shows common terms extracted from all individuals’ ABOUT pages. Some tagged words are part of hyphenated phrases, e.g. co-president. We do not consider common prefixes (e.g. vice-, ex-) and suffixes (e.g. -elect, -in-law) as individual roles during analysis.

heading: F.3 Occupation hierarchy
text: Some of the roles we analyze are occupations, which we define as job titles grouped by the Occupational Information Network, or O*NET, which is created by the U.S. Department of Labor (Table 17). Job titles for occupations listed in O*NET are obtained from three sources. First, occupation pages themselves contain example job titles in singular form, usually in a comma-separated list. Second, the names of occupations often refer to job titles, e.g. Plasterers and Stucco Masons, though in plu-
Learning rate Further pretraining? Precision Recall F1
ral. We singularize and parse these occupation names into job titles by querying GPT-3.5 with the prompt template shown in Figure 9. We manually verify answers from GPT-3.5 that do not agree with a simple rule-based approach of splitting occupations on commas and and and removing -s from terms in occupation titles. Finally, we obtain additional job titles for each O*NET occupation by including all job titles listed in O*NET’s file of “alternate” or “lay” occupational titles.11
11https://www.onetcenter.org/dictionary/20.3/ text/alternate_titles.html
Since English tends to have head-final noun phrases, we attach ABOUT pages to job titles if their last token is classified as a role. Social roles have varying levels of granularity and one term can link to multiple, e.g a floral designer is both a designer due to its head token and a florist due to O*NET. Some commonly extracted social roles (e.g. student, mom) are not in O*NET, so we analyze scores for each of these terms individually. Other terms are ambiguous as to which O*NET
occupation they refer to (e.g. a researcher could be a historian or a geneticist), and so we analyze these individually as well.

heading: F.4 Additional filtering results
text: Table 14 shows an extended version of Table 6.12
For our prestige and salary analyses, we gather salary estimates from O*NET occupation pages, and prestige ratings from Hughes et al. (2022). For ambiguous O*NET job titles (e.g. researcher), we assign them the average salary and prestige of all occupations they belong to. One limitation of this metadata is that these salary and prestige estimates are gathered from a U.S.-centric perspective, and may not generalize to other geographic contexts. Out of all 780 unique social roles that occur more than 1K times in AboutMe, 462 (59.2%) have prestige values and 497 (63.7%) have salaries.
We find that for salary, two filters (WIKIppl and Gopher) show statistically significant relationships with salary, where higher-paid occupations are filtered less (Table 18, Figure 10). For prestige, all quality filters except for Gopher show a statistically significant relationship. The most and least filtered social roles shown in Table 14 intuitively reflect these trends. For example, tech-related engineering occupations are highly scored by OPENWEB, and these tend to have prestige scores over 60.

heading: G Geographic locations

heading: G.1 Geoparsing annotation & evaluation
text: For annotation of geographic locations, we verify, correct, or add to Mordecai3’s predictions. Mordecai3 links mentioned locations to unique IDs in the GeoNames geographic database. We divided 200 randomly sampled ABOUT pages among authors to annotate, and follow annotation instructions shown in Figures 12 and 13. We use the context surrounding a mentioned location and pragmatic principles when making judgements, especially when exact locations are underspecified (Grice, 1975). To calculate interannotator agreement, 35 of 200 pages were doubly annotated. For assigning GeoName IDs, our annotators achieve high pairwise agreement on spans (Cohen’s κ = 0.809). We also annotate whether subjects may identify with mentioned locations on their ABOUT page. Our agreement on this binary task is lower than that of GeoName IDs,
12Though baby showing up as a self-identified role may seem unusual, it occurs in contexts such as I’m an 80s baby.
likely due to the more subjective and interpretive nature of the task (Cohen’s κ = 0.652).
Table 20 outlines the performance of the geoparser we apply onto ABOUT pages. Overall performance is hurt by imperfect recall of spans and accuracy on our data is lower than the country level accuracy of 94.2% reported in the Mordecai3 paper, which mostly trained and evaluated on news and Wikipedia data (Halterman, 2023). By aggregating results to the most frequent country at the page-level, we are able to better navigate errors that may occur at more granular levels. Still, we encourage future work to continue improving geoparsing performance, especially for a wide range of textual domains. Table 19 shows a more extensive overview of count statistics for countries, subregions, and regions present in AboutMe. Out of all pages, 79.5% identify with the most frequent country geoparsed from locations on the page. We also considered taking the country geoparsed from the first span of the page, but only 65.2% of pages were labeled to identify closely with this country.

heading: G.2 Country metadata
text: For our analyses, we incorporate the following metadata for countries: continental region, subregion, gross domestic product (in USD), and anglophone status.
Continental regions and subregions. We use regions and subregions delineated by the United Nations’s “Standard Country or Area Codes for Statistical Use.”13. We add Taiwan to Eastern Asia and Kosovo to Southern Europe, as Mordecai3 produces these country codes. We exclude Antarctica from analysis, as there are less than three hundred webpages geoparsed to it. Since the UN subregions Polynesia, Melanesia, and Micronesia are infrequent in AboutMe, we group them into a single subregion of Pacific Islands.14 This way, all included subregions contain at least 4k websites (Table 19).
Gross domestic product (GDP). Following Zhou et al. (2022), we gather GDP for each country from the World Bank.15 We take the value from most recent listed year where GDP in USD is recorded, which is typically 2022 or 2021.
13https://unstats.un.org/unsd/methodology/m49/ 14https://www.britannica.com/place/ Pacific-Islands 15https://data.worldbank.org/indicator/NY.GDP. MKTP.CD
Anglophone status. The concept of an “Englishspeaking” country can be defined in a variety of ways. Official adoption of English does not necessarily entail high frequency of English use in a country, and vice versa (Plonski et al., 2013). For example, the United States has no official language, yet has a large majority of English speakers. Central to theories around the English-speaking world is that a few countries make up the “core anglosphere”: the United States, Canada, the United Kingdom, Australia, and New Zealand (Vucetic, 2020). We bucket countries into four categories: “core” anglophone, English is an official and primary language, English is an official but not primary language, and all others. We use information about countries’ official and primary language status aggregated on Wikipedia.16

heading: G.3 Additional filtering results
text: We limit country-level filtering analyses only to countries that appear at least 500 times in our dataset, to ensure the patterns we find are over enough samples. We find weak Pearson correlations (p < 0.05, with Bonferroni correction) between a country’s GDP and their average filtering scores for fastText, CLD3, and WIKIppl. However, these results are only due to a single outlier, China, which is often the most filtered country but also very high in GDP. After removing this outlier, all p values are insignificant, and thus we do not confidently conclude any broad relationship between wealth status and filtering.
16https://en.wikipedia.org/wiki/List_of_ countries_and_territories_where_English_is_ an_official_language
In addition, we observe considerable overlap in filtering scores across the four levels of “Englishspeaking” countries (Appendix G.2). This finding, which is contrary to our hypothesis that filters may favor English-speaking locations, suggests that other factors may be at play aside from geography. For example, the topic travel,tours is the most common cluster (9.03%) of websites associated with Northern Africa, and travel websites may be written for outsider audiences and not reflect local communication patterns. Indeed, as our results in Appendix H show, topic usually has a higher and more significant influence on filtering scores than geography-related features.

heading: H Regression
text: In §4.5, we run nine ordinary least squares regressions, one for each model-based filter, to investigate how different aspects of websites that we extract relate to filtering scores. To transform categorical variables into dummy binary variables, we use Africa as the base category for region, and art, gallery as the base category for topical interests. Since the directionality of how WIKIppl should be interpreted is the opposite of other filters’ scores, we negate its scores before performing its regression. Tables 21-29 show the results of these regressions in more detail. For clarity of interpretation, we include coefficients for only a subset of all topics with the most positive and negative effects in each regression. The topics with highest and lowest coefficients tend to reflect ones that are highly retained or removed by a filter.
Occupation family Count Examples of extracted roles
Arts, Design, Entertainment, Sports, & Media 1.1M artist, director, designer, writer, photographer, musician, player
Production 620K designer, engineer, maker, builder, operator, mechanic
Community & Social Service 452K therapist, educator, advisor, pastor, activist, social worker
Computer & Mathematical 365K engineer, developer, scientist, strategist, programmer
Educational Instruction & Library
308K teacher, professor, lecturer, curator, tutor, graduate student
Healthcare Practitioners and Technical 300K therapist, nurse, doctor, nutritionist, surgeon, midwife
Management 291K president, manager, dean, administrator, medical director
Architecture and Engineering 288K architect, technician, electrical engineer, technologist, tester
Business and Financial Operations
250K analyst, accountant, marketer, investor, management consultant
Personal Care and Service 205K trainer, yoga teacher, stylist, makeup artist, caregiver
Table 17: Ten most common O*NET occupation families in AboutMe, by website count, with example social roles. This is an extended version of Table 2.
Filter Salary Prestige
fastText 0.0554 -0.0178 CLD2 -0.0007 -0.0699 CLD3 0.1302 0.0339 langdetect 0.1353 0.0833 WIKIppl 0.2102*** 0.2176*** WIKI 0.1051 0.2336*** WIKIREFS 0.1076 0.1713** OPENWEB 0.1349 0.2280*** WIKIWEBBOOKS 0.1115 0.2238*** Gopher 0.2071*** 0.1139
Table 18: Pearson correlation values between prestige or salary and filters’ scores (higher = less filtered). For Gopher, we use the negated rate of page removal as the “score” since that filter does not output a single numerical score. We also negate scores for WIKIppl so that its values can be interpreted similarly to other rows, since higher perplexities values get filtered more rather than less. Significance: *p < 0.05, **p < 0.01, ***p < 0.001, with Bonferroni correction for 20 comparisons.

heading: Future Work
text: Still, we encourage future work to continue improving geoparsing performance, especially for a wide range of textual domains. Table 19 shows a more extensive overview of count statistics for countries, subregions, and regions present in AboutMe. Out of all pages, 79. 5% identify with the most frequent country geoparsed from locations on the page. We also considered taking the country geoparsed from the first span of the page, but only 65. 2% of pages were labeled to identify closely with this country