MC-19: A Corpus of 19th Century Icelandic Texts (2025)

Abstract: We present MC-19, a new Icelandic historical corpus containing texts from the period 1800-1920. We describe approaches for enhancing a corpus of historical texts, by preparing the texts so that they can be processed using state-of-the-art NLP tools. We train encoder-decoder models to reduce the number of OCR errors while leaving other orthographical variation be. We generate a separate modern spelling layer by normalizing the spelling to comply with modern spelling rules, using a statistical modernization ruleset as well as a dictionary of the most common words. This allows for the texts to be PoS-tagged and lemmatized using available tools, facilitating usage of the corpus for researchers and language technologists. The published version of the corpus contains over 270 million tokens.

## introduction for most areas of language technology, large text corpora and other textual resources have become increasingly important in recent years, not least due to large language models (llms) becoming ever more pervasive. textual resources are not only necessary to train such models to use and decipher language, but also for question answering, information extraction and other generative tasks. with better access to data and tools to work with linguistic data, data-oriented approaches to linguistic research and lexicography have become more common and more useful, allowing more researchers to use such approaches in their work. most commonly, large text corpora comprise recent texts. texts from the digital era, written to be published online, can be a good tool to study recent changes and variation in language, as well as recent events and how they are perceived as they are happening. when we want to study older language, the new methods fall short if the data is lacking. in order to facilitate linguistic research for older texts, we have compiled a new corpus, the 19th century megacorpus . such research might include diachronic linguistic studies and syntactic analysis. the aim of the mc-19 project is to compile as large a corpus as possible, comprising texts written from 1800 to 1920. the first edition of the corpus contains texts from journals and newspapers published in this period and scanned by the national and university library of iceland (lbs), but we intend to extend the corpus in a later edition to also include published books. we use the ocred texts published by lbs and develop postprocessing models to find and fix ocr errors in the texts, while aiming to not change anything else. finally, we normalize the texts using modern spelling. the contributions of the project, presented in this paper, include: syntacticians studying icelandic syntax, linguists studying word formation, inflectional morphology or semantics and lexicographers compiling dictionaries have been the most active users of the icelandic gigaword corpus (igc, @xcite . amongst these users there has been a call for corpora covering larger periods and going as far back in time as possible, in order to further the study of, for example, semantic or syntactic change. language technologists, working on llms, are interested in studying how different llms comprehend older language in comparison with current language and to add older texts into the training process to see if it enhances the models' abilities to generate informative texts covering previous time periods. with mc-19, we aim to facilitate work in all these different fields of research. as a demonstration of research that could be furthered with a corpus like ours, we could look at an empirical study on the reflexive passive in icelandic conducted by @xcite . this construction can be dated back to the 19th century as @xcite show. to find as old examples as they could at tímarit.is, the authors had to look for word strings. to find different examples of flýta sér 'hurry (oneself)' in the reflexive passive, they had to search for, e.g., "var flýtt sér" ('was hurried oneself'), "var flýtt sjer", "er flýtt sér", "er flýtt sjer", "verið flýtt sér", "verið flýtt sjer", etc.; they also searched for, e.g., adverbs like oft 'often' intervening between the auxiliary vera 'be' and the participle (cf. @xcite . this is rather time consuming, especially when one wants to look for as many different verbs as possible. this is, however, made easier in mc-19 as the corpus is pos-tagged and lemmatized and we can therefore look for both certain word forms and tags. a search query that looks for the lemma vera 'be' followed by past participle (and between vera and the participle can be at most one word) which in turn is immediately followed by the reflexive pronoun forms sig/sér/sín seems to return most of the 19th-century examples from árnadóttir et al.'s study (but of course not the ones that differ in structure from the setup in the query). this search query also returns at least two exam-ples from the 19th century that are not reported on in @xcite . ## related work a wide range of historical corpora has been compiled and made available for different languages. many of these are small, less than a million words, but there are notable exceptions. the corpus of late modern english texts @xcite contains over 34 million words in texts from the period 1710-1920, and the royal society corpus @xcite includes all publications of the philosophical transactions of the royal society of london from 1665 to 1869, approximately 32 million tokens. chroniclitaly @xcite is a corpus of italian language newspapers published in the united states between 1898 and 1920, 16.6 million words in total, and the diorisis ancient greek corpus contains 10.2 million words in texts spanning from homer to the fifth century ad @xcite . turning to icelandic, the icelandic parsed historical corpus (icepahc, @xcite ) contains approximately 1 million words written between the 12th and 21st centuries. the saga corpus @xcite contains the texts of the icelandic sagas as well as a few other historical texts in modernized editions, and the igc, which is 2.6 billion words in total and mostly has texts from the 21st century and the end of the 20th century, contains a few thousand words in texts written before the year 1900, all from the igc-law @xcite subcorpus, containing law texts. a number of studies have been carried out on how best to correct historical ocr data. @xcite present a pipeline for correcting 19th century danish fraktur. their approach is rather different from ours, starting by changing "obvious and unambiguous ocr errors", then aligning multiple ocr output candidates and perform selective correction with reference to these and finally employing a spell checker. different approaches have been taken when doing historical spelling normalization. @xcite use machine translation (mt) systems, translating original spelling into normalized texts. while they compare rule-based and smtbased mt systems, @xcite evaluate the effectiveness of using neural-based mt for the task. @xcite highlights that there is no consensus on the state-of-the-art approach to historical text normalization and compares a number of approaches. he finds that lookups based on naive memorization are most often effective for seen tokens, while mt-based methods perform best in unseen cases. ## data processing our data is collected from tímarit.is, a digital library platform for newspapers and periodicals that goes back to the early 19th century. the platform allows users to search texts, with ocr-generated text files for each page in the library. rather than running our own ocr-models on the pages, which would have been resource intensive and not necessarily very beneficial, we decided to use the texts ocred by the providers of tímarit.is, lbs. in order to facilitate our work, lbs provided us with all text files for our project, covering the period in question, 1800-1920. so that we could exclude too noisy texts, we manually checked the ocr quality of newspapers and periodicals that were candidates for our corpus. the process is described in section 4.1. during the selection process we compiled a list of common ocr errors. we then enlarged it by extracting a list of ocr errors from manually corrected texts from this period that we had access to. the information was used to automatically introduce ocr-like errors to correct texts, thus creating a parallel data set for training models to postprocess ocred data. we also took random samples from the texts that we decided to use and manually fixed the ocr errors to create an evaluation set. we describe this in more detail in section 4.2. all the selected texts were run through the postprocessing models we trained, before normalizing them to modern spelling, using the approaches described in section 4.3. having the modern spelling variants we could pos-tag and lemmatize the texts using the best available tools for icelandic, which are trained on modern texts. ## use and availability the corpus is published under an open cc by 4.0 license. it is available online in two different forms for different uses and users. it is made available for search online in a kwic-portal, powered by korp @xcite . users can search for word forms in both the original version (ocred text) and in the modern spelling transcription, with the modern spelling transcription being pos-tagged and lemmatized, allowing for more complicated search in that data. the results are shown in parallel, so while the user can search using modern spelling varieties, the original ones are also shown. this format is expected to mostly be useful to linguists, lexicographers and students of icelandic. the tei-version is available for download. it contains whole sentences in the original version as well as the normalized version using modern spelling. the normalized version is furthermore pos-tagged and lemmatized. we expect this format to be most useful for language technologists for analyzing and building tools and language models. linguists competent in programming may also find that working with these annotated documents allows for more complicated analysis and research than when limited to kwicanalysis. ## conclusion and future work we have presented a new text corpus, mc-19, containing icelandic texts from the 19th century and the first decades of the 20th century. the first version of this corpus has been published and is made available in a tei-format as well as in an online kwic-platform, powered by korp. while care has been taken to make the texts as readable and close to the printed material as possible, using a post-processing step and a spellingmodernization step, there is always room for improvement. the post-processing process reduces the number of ocr errors by 55.07%. improving the performance in this step would make the corpus more accurate and useful. this could possibly be achieved by improving the post-processing models, for example by generating more artificial training data or more diverse training data. some error reduction may be achieved simply by replacing possible errors with possible corrections, using our error list. for such an approach, which tends to be greedy, some measures would need to be taken to limit the possibility of generating new errors. this could possibly be achieved by mapping only from unknown words (containing possible errors) to known words, calculating the likelihood of the change using n-grams or perplexity calculations or other approaches that may prove useful. while most of the sentences in the corpus are as printed in the original publications, some are garbled due to problems with ocr that our methods could not solve. training a classifier to select bad sentences for removal could make the corpus an even better tool. the spelling-modernization step helps the user find common words for which the spelling has changed, allowing for easier search and usage of the corpus, but the user will still find that some words are not modernized. a more thorough examination of this and improvements in the process will help with using the corpus for research. we intend to revisit these steps for a future version of the corpus, integrating additional normalization techniques and manually evaluate the merits of different approaches to this problem. we also intend to add texts from books published in the period, and are working on ocr-reading fraktur texts. while these texts may not add very much to this corpus in terms of word count, as the bulk of published texts in the period is in newspapers and periodicals, it may show a greater variety, both in terms of language and content. available texts from previous periods, printed and hand-written, are also being considered for a sister corpus to this one.