Intersecting Register and Genre: Understanding the Contents of Web-Crawled Corpora (2024)

Abstract: Web-scale corpora present valuable research opportunities but often lack detailed metadata, making them challenging to use in linguistics and social sciences. This study tackles this problem by exploring automatic methods to classify web corpora into specific categories, focusing on text registers such as Interactive Discussion and literary genres such as Politics and Social Sciences. We train two machine learning models to classify documents from the large web-crawled OSCAR dataset: a register classifier using the multilingual, manually annotated CORE corpus, and a genre classifier using a dataset based on Kindle US&UK. Fine-tuned from XLM-R Large, the register and genre classifiers achieved F1-scores of 0.74 and 0.70, respectively. Our analysis includes evaluating the distribution of the predicted text classes and examining the intersection of genre-register pairs using topic modelling. The results show expected combinations between certain registers and genres, such as the Lyrical register often aligning with the Literature & Fiction genre. However, most registers, such as Interactive Discussion, are divided across multiple genres, like Engineering & Transportation and Politics & Social Sciences, depending on the discussion topic. This enriched metadata provides valuable insights and supports new ways of studying digital cultural heritage.

## introduction automatically collected web-scale corpora, encompassing billions of words, offer significant opportunities for research across a range of disciplines, including computational linguistics, natural language processing, social sciences, and digital humanities. these extensive resources have been, and continue to be, instrumental in advancing large language models, such as the one underpinning chatgpt. additionally, these corpora contain vast amounts of text produced in varied contexts and for diverse purposes, serving as repositories for new and evolving forms of digital cultural heritage. consequently, web-scale corpora not only open new research avenues in the humanities and social sciences but also necessitate multidisciplinary collaboration to ensure their effective use @xcite . a notable challenge in utilizing web-scale corpora is the lack of detailed metadata describing their contents. without such metadata, texts of diverse varieties-such as legal notices, advertisements, news articles, fiction, and song lyrics-are treated equally, despite their distinct linguistic characteristics. this complicates the filtering and selection of data relevant to specific research tasks. among others, these procedures are often crucial for building accurate language models, where the absence of metadata increases the risk of models learning from biased, toxic, or irrelevant data (e.g. @xcite @xcite @xcite . text type metadata has also been shown to enhance the performance of various natural language processing (nlp) applications, including part-of-speech taggers, parsers, and information retrieval systems (e.g. @xcite @xcite @xcite . to address this challenge, we explore a dual approach to classifying web corpora into specific text categories, focusing on two key approaches: register and genre. registers, as they are typically applied in corpus linguistics, refer to culturally recognizable text varieties characterized by their communicative situation and functionally related linguistic features @xcite @xcite . genres, in literary studies, are often utilized to examine various forms of literary work, focusing on content, context, and narrative tools (e.g., @xcite . in recent years, text classification and specifically web register identification has taken leaps forward, with web register classifiers achieving nearly human-level performance @xcite @xcite . these advancements enable us to enhance document metadata substantially. however, when register classification is applied to web-scale corpora, the predicted register classes are still very broad and include a wide range of linguistic variation. therefore, in this study, we examine whether combining two approaches, namely registers and genres, can enhance the depth of the available information for a given document. specifically, we examine the intersections between register and genre labels in a text classification setting and explore how these intersections, along with the resulting new metadata, can allow for novel uses of the corpus in other studies. to achieve this, we apply machine learning to train two text classifiers: one targeting registers and the other focusing on genres. these classifiers are then used to predict classes for one million documents from the widely used web to assess the quality of the new metadatacombined label of register and genre classifiers-we evaluate two conditions. first, we examine the overlap between the two labelling schemes, as crosslabelling has little value if the classifiers produce similar results. ideally, each register class should map onto multiple genre categories, adding previ-ously unattainable information to each document. second, we evaluate whether these cross-labellings are meaningful by using topic modelling to extract topic words for each register-genre combination. ## data we use three different datasets for our experiments, namely the corpus of online registers of english (core), the genre-6 literary genre corpus, and the open super-large crawled almanach corpus (oscar), each for a different task. ## experimental setup figure 1 illustrates the workflow of the classification process. we use two classifiers to predict labels for each document from the oscar corpus, and the resulting labels are then combined into a single, combined label. ## conclusion in this study, we experimented with labelling a large internet corpus using two classifiers and evaluated the new metadata produced by the intersection of two classification schemes. we trained the classifiers on available register and genre datasets in a multilabel classification setting and reached reliable results. we then analyzed the distributions of the intersection classes and extracted topic keywords from them. our evaluation, based on quantitative analysis using topic modelling and close reading, demonstrated that the predicted genre and register labels provide meaningful auxiliary information, facilitating new ways to use the corpus. this is particularly valuable for digital humanities and cultural heritage studies, as it allows for richer contextualization and more nuanced analysis of historical documents, literary texts, and other cultural artefacts. enhanced metadata can also support the preservation and accessibility of digital archives, ensuring that documents are more easily discoverable and interpretable. in the future, we will aim to improve model performance by experimenting with different model architectures and refining the chosen classes of the genre classifier. specifically, we recognize that the current literature & fiction category is too broad and plan to subdivide it into more specific genres. we are also interested in exploring different data augmentation techniques (e.g. label cleaning tools foot_5 ), particularly for our genre corpus, which has shown apparent label issues during manual review.