Update README.md

README.md

@@ -16,8 +16,7 @@ I undertook the following preprocessing steps:
   - Language identification with [langid](https://github.com/saffsd/langid.py)
   - Compute the ratio of Latin vocabulary in each sentence (against the digital-born vocab of the corpus)
   - Retain only sentences with a Latin vocabulary ratio of > 85%.
-  - Retaining only lines containing letters of the Latin alphabet, numerals, and certain punctuation (--> `grep -P '^[A-z0-9ÄÖÜäöüÆ挜ᵫĀāūōŌ.,;:?!\- Ęę]+$' la.nolorem.tok.txt`
-  - deduplication of the corpus
+  - Exclude all lines containing '^' --> hints at the presence of OCR errors.
 
 The result is a corpus of ~390 million tokens.