Repository: flairNLP/flair

Issue:
FlairEmbeddings function gives ValueError in python 3.6 (latest Nvidia Pytorch Docker container)
**Describe the bug**
Simply running the code ```FlairEmbeddings('news-forward')``` gives a ```ValueError```, in Python 3.6.10 (Conda), which is the python environment included in the most recent PyTorch Docker container from Nvidia. (https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/rel_20-06.html#rel_20-06)

Here is the error message:
```
Traceback (most recent call last):
  File "fineTune_langModel.py", line 10, in <module>
    language_model = FlairEmbeddings('news-forward').lm
  File "/opt/conda/lib/python3.6/site-packages/flair/embeddings/token.py", line 578, in __init__
    self.lm: LanguageModel = LanguageModel.load_language_model(model)
  File "/opt/conda/lib/python3.6/site-packages/flair/models/language_model.py", line 202, in load_language_model
    dropout=state["dropout"],
  File "/opt/conda/lib/python3.6/site-packages/flair/models/language_model.py", line 63, in __init__
    self.to(flair.device)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 465, in to
    return self._apply(convert)
  File "/opt/conda/lib/python3.6/site-packages/flair/models/language_model.py", line 404, in _apply
    for info in torch.__version__.replace("+",".").split('.') if info.isdigit())
ValueError: not enough values to unpack (expected at least 3, got 2)
```


**To Reproduce**
Run this code in the container (after pip installing flair):
```from flair.embeddings import FlairEmbeddings```
```FlairEmbeddings('news-forward')```

**Expected behavior**
The function should work with no problems (in particular, the .lm is intended to be given to the LanguageModelTrainer function).

**Environment (please complete the following information):**
- Docker container running Linux (all details of Ubuntu, Python, PyTorch, etc. versions is in the Nvidia link above)
- Flair version is 0.5

**Additional context**
Running the code in Python 3.7.7 (Conda) gives no problems.


Discussion:
@tylerlekang thanks for reporting this. Could you print the torch version you get with `torch.__version__`? Also, can you try updating to Flair 0.5.1?


@alanakbik ```torch.__version__``` reports ```1.6.0a0+9907a3e``` , which matches as shown in https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/rel_20-06.html#rel_20-06

Used ```pip install --upgrade flair``` to upgrade to 0.5.1. The same error persists:
```
>>> flair.__version__
'0.5.1'
>>>
>>> from flair.embeddings import FlairEmbeddings
>>> FlairEmbeddings('news-forward')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/lib/python3.6/site-packages/flair/embeddings/token.py", line 586, in __init__
    self.lm: LanguageModel = LanguageModel.load_language_model(model)
  File "/opt/conda/lib/python3.6/site-packages/flair/models/language_model.py", line 202, in load_language_model
    dropout=state["dropout"],
  File "/opt/conda/lib/python3.6/site-packages/flair/models/language_model.py", line 63, in __init__
    self.to(flair.device)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 465, in to
    return self._apply(convert)
  File "/opt/conda/lib/python3.6/site-packages/flair/models/language_model.py", line 404, in _apply
    for info in torch.__version__.replace("+",".").split('.') if info.isdigit())
ValueError: not enough values to unpack (expected at least 3, got 2)
```

Did you test with this container? It seems like a common and important container to verify, as it is official Nvidia optimized for PyTorch applications.

Thank you very much for your support! :)
@alanakbik  In the models/language_model.py code, the first line of _apply is (starts at line 402):
```
major, minor, build, *_ = (int(info)
                                for info in torch.__version__.replace("+",".").split('.') if info.isdigit())
```

If I simply run ```torch.__version__.replace("+",".").split('.')``` in the container (python 3.6.10) it returns ```['1', '6', '0a0', '9907a3e']```. Then if I run ```for i in (int(info) for info in torch.__version__.replace("+",".").split('.') if info.isdigit()): print(i)``` it prints:
```
1
6
```

However, on my local machine running vanilla python 3.7.7, the torch version is just ```1.5.0```. So this may be the problem.

I have no idea why Nvidia has chosen this version of Pytorch with letters in the version number, but they did make this choice and this container is supposed to be an easy solution for highly optimized GPU runs on their hardware.

Do you have any workaround ideas?
@alanakbik could I just hardcode the major, minor, build numbers, in my local version of language_model.py if there is no workaround? 

```
major = 1
minor = 6
build = 0
```

It seems the code just checks that the major.minor is >= 1.4 ? But I don't want to mess up any other parts of the code.
Yes, I guess you could just overwrite `torch.__version__` as a quick fix by calling this before your script: 

```python
import torch

torch.__version__ = '1.5.0'
```

Meanwhile, I will put in a PR to fix the error.
@alanakbik just wanting to triple-confirm, that shouldn't cause any problems with the rest of the FlairEmbeddings or LanguageModelTrainer codes? Thank you!
It shouldn't cause any problems on the flair side. We use the string to determine whether and old version of torch is used (<1.4.0) or not, so changing it to another string that is above 1.4.0 won't change anything. 


Resolving patch:
diff --git a/flair/embeddings/document.py b/flair/embeddings/document.py
--- a/flair/embeddings/document.py
+++ b/flair/embeddings/document.py
@@ -444,38 +444,32 @@ def _add_embeddings_internal(self, sentences: Union[List[Sentence], Sentence]):
             sentence.set_embedding(self.name, embedding)
 
     def _apply(self, fn):
-        major, minor, build, *_ = (int(info)
-                                for info in torch.__version__.replace("+",".").split('.') if info.isdigit())
-
-        # fixed RNN change format for torch 1.4.0
-        if major >= 1 and minor >= 4:
-            for child_module in self.children():
-                if isinstance(child_module, torch.nn.RNNBase):
-                    _flat_weights_names = []
-                    num_direction = None
-
-                    if child_module.__dict__["bidirectional"]:
-                        num_direction = 2
-                    else:
-                        num_direction = 1
-                    for layer in range(child_module.__dict__["num_layers"]):
-                        for direction in range(num_direction):
-                            suffix = "_reverse" if direction == 1 else ""
-                            param_names = ["weight_ih_l{}{}", "weight_hh_l{}{}"]
-                            if child_module.__dict__["bias"]:
-                                param_names += ["bias_ih_l{}{}", "bias_hh_l{}{}"]
-                            param_names = [
-                                x.format(layer, suffix) for x in param_names
-                            ]
-                            _flat_weights_names.extend(param_names)
-
-                    setattr(child_module, "_flat_weights_names",
-                            _flat_weights_names)
-
-                child_module._apply(fn)
 
-        else:
-            super()._apply(fn)
+        # models that were serialized using torch versions older than 1.4.0 lack the _flat_weights_names attribute
+        # check if this is the case and if so, set it
+        for child_module in self.children():
+            if isinstance(child_module, torch.nn.RNNBase) and not hasattr(child_module, "_flat_weights_names"):
+                _flat_weights_names = []
+
+                if child_module.__dict__["bidirectional"]:
+                    num_direction = 2
+                else:
+                    num_direction = 1
+                for layer in range(child_module.__dict__["num_layers"]):
+                    for direction in range(num_direction):
+                        suffix = "_reverse" if direction == 1 else ""
+                        param_names = ["weight_ih_l{}{}", "weight_hh_l{}{}"]
+                        if child_module.__dict__["bias"]:
+                            param_names += ["bias_ih_l{}{}", "bias_hh_l{}{}"]
+                        param_names = [
+                            x.format(layer, suffix) for x in param_names
+                        ]
+                        _flat_weights_names.extend(param_names)
+
+                setattr(child_module, "_flat_weights_names",
+                        _flat_weights_names)
+
+            child_module._apply(fn)
 
 
 class DocumentLMEmbeddings(DocumentEmbeddings):
diff --git a/flair/models/language_model.py b/flair/models/language_model.py
--- a/flair/models/language_model.py
+++ b/flair/models/language_model.py
@@ -400,35 +400,29 @@ def calculate_perplexity(self, text: str) -> float:
         return perplexity
 
     def _apply(self, fn):
-        major, minor, build, *_ = (int(info)
-                                for info in torch.__version__.replace("+",".").split('.') if info.isdigit())
-
-        # fixed RNN change format for torch 1.4.0
-        if major >= 1 and minor >= 4:
-            for child_module in self.children():
-                if isinstance(child_module, torch.nn.RNNBase):
-                    _flat_weights_names = []
-                    num_direction = None
-
-                    if child_module.__dict__["bidirectional"]:
-                        num_direction = 2
-                    else:
-                        num_direction = 1
-                    for layer in range(child_module.__dict__["num_layers"]):
-                        for direction in range(num_direction):
-                            suffix = "_reverse" if direction == 1 else ""
-                            param_names = ["weight_ih_l{}{}", "weight_hh_l{}{}"]
-                            if child_module.__dict__["bias"]:
-                                param_names += ["bias_ih_l{}{}", "bias_hh_l{}{}"]
-                            param_names = [
-                                x.format(layer, suffix) for x in param_names
-                            ]
-                            _flat_weights_names.extend(param_names)
-
-                    setattr(child_module, "_flat_weights_names",
-                            _flat_weights_names)
-
-                child_module._apply(fn)
 
-        else:
-            super()._apply(fn)
+        # models that were serialized using torch versions older than 1.4.0 lack the _flat_weights_names attribute
+        # check if this is the case and if so, set it
+        for child_module in self.children():
+            if isinstance(child_module, torch.nn.RNNBase) and not hasattr(child_module, "_flat_weights_names"):
+                _flat_weights_names = []
+
+                if child_module.__dict__["bidirectional"]:
+                    num_direction = 2
+                else:
+                    num_direction = 1
+                for layer in range(child_module.__dict__["num_layers"]):
+                    for direction in range(num_direction):
+                        suffix = "_reverse" if direction == 1 else ""
+                        param_names = ["weight_ih_l{}{}", "weight_hh_l{}{}"]
+                        if child_module.__dict__["bias"]:
+                            param_names += ["bias_ih_l{}{}", "bias_hh_l{}{}"]
+                        param_names = [
+                            x.format(layer, suffix) for x in param_names
+                        ]
+                        _flat_weights_names.extend(param_names)
+
+                setattr(child_module, "_flat_weights_names",
+                        _flat_weights_names)
+
+            child_module._apply(fn)
\ No newline at end of file


Test patch: