Update README.md

README.md

@@ -20,45 +20,51 @@ widget:
 <img src="https://upload.wikimedia.org/wikipedia/commons/5/5b/NCI_peas_in_pod.jpg" alt="erwt" width="200" >
 
 # ERWT-year
-\~🌺\~A language model that is (🤭 maybe 🤫) better at history than you...\~🌺\~
+\~🌺\~ERWT**\*** is a language model that is (🤭 maybe 🤫) better at history than you...\~🌺\~
 
+\*ERWT is dutch for PEA.
 
-ERWT is a fine-tuned [`distilbert-base-cased`](https://huggingface.co/distilbert-base-cased) model trained on historical newspapers from the [Heritage Made Digital collection](https://huggingface.co/datasets/davanstrien/hmd-erwt-training) with temporal metadata. 
+ERWT is a fine-tuned [`distilbert-base-cased`](https://huggingface.co/distilbert-base-cased) model trained on historical newspapers from the [Heritage Made Digital collection](https://huggingface.co/datasets/davanstrien/hmd-erwt-training) with **temporal metadata**.
+
+ERWT performs time-sensitive masked language modelling. It can also guess the year a text was written.
 
 This model is served you by [Kaspar von Beelen](https://huggingface.co/Kaspar) and [Daniel van Strien](https://huggingface.co/davanstrien).
 
 *Improving AI, one pea at a time.*
 
-
 ## Introductory Note: Repent Now. 😇
 
-This model was trained for **experimental purposes**, please use it with care. 
+The ERWT models are trained for **experimental purposes**, please use it with care. 
 
+You find more detailed information below. Please consult the **limitations** section (seriously, read this section before using the models, **we don't repent in public just for fun**). 
 
-You find more detailed information below, especially the "limitations" section. Seriously, read this section before using the models, we don't repent in public just for fun. 
-
-If you can't get enough, you can still consult our working paper ["Metadata Might Make Language Models Better"](https://drive.google.com/file/d/1Xp21KENzIeEqFpKvO85FkHynC0PNwBn7/view?usp=sharing) for more background and nerdy evaluation stuff (still, work in progress, handle with care and kindness).
+If you can't get enough of these peas and crave for some more, you can still consult our working paper ["Metadata Might Make Language Models Better"](https://drive.google.com/file/d/1Xp21KENzIeEqFpKvO85FkHynC0PNwBn7/view?usp=sharing) for more background information and nerdy evaluation stuff (work in progress, handle with care and kindness).
 
 ## Background: MDMA to the rescue. 🙂
 
-ERWT was created using a **M**eta**D**ata **M**asking **A**pproach (or **MDMA** 💊), in which we train a Masked Language Model simultaneously on text and metadata. Our intuition was that incorporating information that is not explicitly present in the text—such as the time of publication or the political leaning of the author—may make language models "better" in the sense of being more sensitive to historical and political aspects of language use.
+ERWT was created using a **M**eta**D**ata **M**asking **A**pproach (or **MDMA** 💊), in which we train a Masked Language Model simultaneously on text and metadata. Our intuition was that incorporating metadata (information that describes a text but and is as the time of publication or the political leaning of the author) may make language models "better", or at least make them more sensitive to historical and political aspects of language use.
+
+ERWT is a [`distilbert-base-cased`](https://huggingface.co/distilbert-base-cased) model, fine-tuned on a random subsample taken from the Heritage Made Digital newspaper collection. The training data comprises around half a billion words. 
 
-To create this ERWT model we fine-tuned [`distilbert-base-cased`](https://huggingface.co/distilbert-base-cased) on a random subsample of the Heritage Made Digital newspaper of about half a billion words. We slightly adapted to the training routine by adding the year of publication and a special token `[DATE]` in front of each text segment (i.e. a chunk of hundred tokens). 
+To unleash the power of MDMA, we adapted to the training routine for the masked language model. When preprocessing the text, we prepended each segment of hundred words with a time-stamp (year of publication) and a special `[DATE]` token. 
 
-For example, we would format a snippet of text taken from the [Londonderry Sentinel](https://www.britishnewspaperarchive.co.uk/viewer/bl/0001480/18700722/014/0002) as...
+The snippet below, taken from the [Londonderry Sentinel](https://www.britishnewspaperarchive.co.uk/viewer/bl/0001480/18700722/014/0002),  
 ```python
 "1870 [DATE] Every scrap of intelligence relative to the war between France and Prussia is now read with interest."
 ```
 
-... and then provide this sentence with prepended temporal metadata to MLM. While training, the model learns a relation between the text and the time it was produced. When a token is masked, the prepended `year` field is taken into account when predicting candidates. But vice versa, if the metadata token at the front of the formatted snippet is hidden, the models aims to predict the year of publication based on the content of a document.
+These formatted chunks of text are then forward to the data collator and eventually the language model.
 
+Exposed to both the tokens and (temporal) metadata, the model learns a relation between text and time. When a token is masked, the prepended `year` field is taken into account when predicting hidden word in the text. Vice versa, when the metadata token at the front of the formatted snippet is hidden, the models aims to predict the year of publication based on the content of a document.
 
-
-## Intended uses 
+## Intended Uses: LMs as History Machines.
 
 Exposing the model to temporal metadata allows us to investigate **historical language change** and perform **date prediction**.
 
-### Language Change 👑
+### Historical Language Change: Her/His Majesty? 👑
+
+
+Let's show a very concrete example.
 
 Also in the nineteenth-century Britain had Queen on the throne for a very long time. Queen Victoria ruled from 1837 to 1901.