Added usage instructions
Browse files
README.md
CHANGED
@@ -8,7 +8,14 @@ tags:
|
|
8 |
|
9 |
|
10 |
|
11 |
-
An ELECTRA-small model for Ancient Greek, trained on texts from Homer up until the 4th century AD.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
12 |
|
13 |
## Citation
|
14 |
|
|
|
8 |
|
9 |
|
10 |
|
11 |
+
An ELECTRA-small model for Ancient Greek, trained on texts from Homer up until the 4th century AD from the literary [GLAUx](https://github.com/alekkeersmaekers/glaux) corpus and the [DukeNLP](https://github.com/alekkeersmaekers/duke-nlp) papyrus corpus.
|
12 |
+
|
13 |
+
The model has some design choices made to combat data sparsity:
|
14 |
+
* Its input should always be in Unicode NFD (so separate Unicode signs for diacritics).
|
15 |
+
* All grave accents should be replaced with acute accents (καί, not καὶ).
|
16 |
+
* When a word contains two accents, the second one should be removed (εἶπε μοι, not εἶπέ μοι).
|
17 |
+
|
18 |
+
If you use it in conjunction with [glaux-nlp](https://github.com/alekkeersmaekers/glaux-nlp), you can pass the tokenized sentence to normalize_tokens from tokenization.Tokenization, using normalization_rule=greek_glaux, which will do all these normalizations for you.
|
19 |
|
20 |
## Citation
|
21 |
|