Repository: flairNLP/flair

Issue:
Add support for Multilingual Flair Embeddings
Hi,

I trained multilingual Flair Embeddings for over 300 languages on the [JW300](https://www.aclweb.org/anthology/P19-1310/) corpus.

The training corpus consists of 2,025,826,977 token (2B) and was trained for 5 epochs. Thanks to half precision support this could be done in 10 days on a V100 😅 (so the total training time for both forward and backward language model was 20 days).

The model can be downloaded from:

```bash
$ wget https://schweter.eu/cloud/flair-lms/lm-jw300-forward-v0.1.pt
$ wget https://schweter.eu/cloud/flair-lms/lm-jw300-backward-v0.1.pt
```

To use it in Flair:

```python
from flair.embeddings import FlairEmbeddings

jw_forward = FlairEmbeddings("lm-jw300-forward-v0.1.pt")
jw_backward = FlairEmbeddings("lm-jw300-backward-v0.1.pt")
```

More details can be found in [this repo](https://github.com/stefan-it/flair-lms#multilingual-flair-embeddings).

Additionally, I made a few experiments for PoS tagging on the Universal Dependencies v1.2 dataset. It seems that this new multilingual language model can achieve comparable results to a multilingual BERT model. More details can be found in [this repository](https://github.com/stefan-it/flair-pos-tagging), including a multilingual evaluation of (mono-lingual) Flair Embeddings.

I would like to thank Željko Agić (@zeljkoagic) for giving me early access to the JW300 corpus  🤗


Discussion:
Bravo!
Made any check NER score on classical datasets? Vs dedicated language Flair LM?
Happy to see our dataset put to good use, nice results! Cite if you use, share alike, you know the drill!
This is awesome @stefan-it! Should we add it as the new default for 'multi-forward' and 'multi-backward'  in `FlairEmbeddings`? I could reupload these models on our server and add this in a PR.
@pommedeterresautee Thanks :) Experiments on WikiANN are definitely on my todo list for autumn 😅

Resolving patch:
diff --git a/flair/embeddings.py b/flair/embeddings.py
--- a/flair/embeddings.py
+++ b/flair/embeddings.py
@@ -1608,10 +1608,12 @@ def __init__(self, model, fine_tune: bool = False, chars_per_chunk: int = 512):
 
         self.PRETRAINED_MODEL_ARCHIVE_MAP = {
             # multilingual models
-            "multi-forward": f"{aws_path}/embeddings-v0.4/lm-multi-forward-v0.1.pt",
-            "multi-backward": f"{aws_path}/embeddings-v0.4/lm-multi-backward-v0.1.pt",
-            "multi-forward-fast": f"{aws_path}/embeddings-v0.4/lm-multi-forward-fast-v0.1.pt",
-            "multi-backward-fast": f"{aws_path}/embeddings-v0.4/lm-multi-backward-fast-v0.1.pt",
+            "multi-forward": f"{aws_path}/embeddings-v0.4.3/lm-jw300-forward-v0.1.pt",
+            "multi-backward": f"{aws_path}/embeddings-v0.4.3/lm-jw300-backward-v0.1.pt",
+            "multi-v0-forward": f"{aws_path}/embeddings-v0.4/lm-multi-forward-v0.1.pt",
+            "multi-v0-backward": f"{aws_path}/embeddings-v0.4/lm-multi-backward-v0.1.pt",
+            "multi-v0-forward-fast": f"{aws_path}/embeddings-v0.4/lm-multi-forward-fast-v0.1.pt",
+            "multi-v0-backward-fast": f"{aws_path}/embeddings-v0.4/lm-multi-backward-fast-v0.1.pt",
             # English models
             "en-forward": f"{aws_path}/embeddings-v0.4.1/big-news-forward--h2048-l1-d0.05-lr30-0.25-20/news-forward-0.4.1.pt",
             "en-backward": f"{aws_path}/embeddings-v0.4.1/big-news-backward--h2048-l1-d0.05-lr30-0.25-20/news-backward-0.4.1.pt",


Test patch: