internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: flairNLP/flair

Issue:
Don't pin package dependencies in setup.py
To be removed, once it is done: Please add the appropriate label to this ticket, e.g. feature or enhancement.

**Is your feature/enhancement request related to a problem? Please describe.**

It is not considered good practice to pin package dependencies in setup.py (see additional context).

For instance, I'm forced to downgrade certain packages by installing flair.

**Describe the solution you'd like**

Just list the abstract requirements in setup.py with less restrictive version bounds.

**Additional context**

See https://packaging.python.org/discussions/install-requires-vs-requirements/


Discussion:
Good point! Care to do a PR? :)

Which packages are you currently forced to downgrade? 
> Care to do a PR? :)

Sure, but I'm not very familiar with flair.

I looked through the dependencies, and `typing` can be removed, since it's a backport for python < 3.5, but you have `python_requires='>=3.6'`.

For the rest, we could replace `==` by `>=`.

Regarding `pytorch-pretrained-bert==0.3.0`, will it work with `0.4.0`, or should that dependency stay pinned?

> Which packages are you currently forced to downgrade?

In my environment I had the latest versions, which caused gensim, tqdm to be downgraded.
I accidentally only updated `pytorch-pretrained-bert` to *0.4.0* in `requirements.txt` and not in `setup.py`, so you can definitely update it to *0.4.0* (I'm using that version in all of my experiments).

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -12,18 +12,17 @@
     packages=find_packages(exclude='test'),  # same as name
     license='MIT',
     install_requires=[
-        'torch==1.0.0',
-        'gensim==3.4.0',
-        'typing==3.6.4',
-        'tqdm==4.26.0',
-        'segtok==1.5.7',
-        'matplotlib==3.0.0',
-        'mpld3==0.3',
+        'torch>=1.0.0',
+        'gensim>=3.4.0',
+        'tqdm>=4.26.0',
+        'segtok>=1.5.7',
+        'matplotlib>=3.0.0',
+        'mpld3>=0.3',
         'sklearn',
-        'sqlitedict==1.6.0',
-        'deprecated==1.2.4',
-        'hyperopt==0.1.1',
-        'pytorch-pretrained-bert==0.3.0'
+        'sqlitedict>=1.6.0',
+        'deprecated>=1.2.4',
+        'hyperopt>=0.1.1',
+        'pytorch-pretrained-bert>=0.3.0'
     ],
     include_package_data=True,
     python_requires='>=3.6',


Test patch:
Repository: flairNLP/flair

Issue:
Cannot install allennlp due to matplotlib dependency conflict
Hello, thanks for the great package. I want to play with ELMoEmbeddings, which requires package allennlp, not installed by default with Flair. However, installing latest allennlp fails because it requires matplotlib==2.2.3, while Flair requires >=3.0.0. When I install allennlp==0.7.0 (without the matplotlib requirement), it still fails because of PyTorch conflicts. Makes me wonder what versions I should use that are actually compatible?



Discussion:
Did you both install `flair` and `allennlp` from latest `master` branch? In my experiments using ELMo within `flair` I didn't get these error messages (installed via `pip install -e .`) 🤔
If you look at the requirements.txt you'll see that the version conflict is in current master branches. And latest releases.
In a clean Miniconda environment there are some messages. Installation via `pip install -e .`, `flair` is installed first:

```bash
boto3 1.7.75 has requirement botocore<1.11.0,>=1.10.75, but you'll have botocore 1.8.36 which is incompatible.
flair 0.4.0 has requirement matplotlib>=3.0.0, but you'll have matplotlib 2.2.3 which is incompatible.
moto 1.3.4 has requirement botocore>=1.9.16, but you'll have botocore 1.8.36 which is incompatible.
```

The newer `matplotlib` version will be uninstalled:

```bash
Found existing installation: wrapt 1.11.1
    Uninstalling wrapt-1.11.1:
      Successfully uninstalled wrapt-1.11.1
  Found existing installation: regex 2019.1.24
    Uninstalling regex-2019.1.24:
      Successfully uninstalled regex-2019.1.24
  Found existing installation: botocore 1.10.84
    Uninstalling botocore-1.10.84:
      Successfully uninstalled botocore-1.10.84
  Found existing installation: matplotlib 3.0.2
    Uninstalling matplotlib-3.0.2:
      Successfully uninstalled matplotlib-3.0.2
  Running setup.py develop for allennlp
```

And `allennlp` can definitely be used then :)
That's a workaround, and probably a non-working setup if you would like to use functionalities using matplotlib from either Flair or AllenNLP. I am using pipenv and cannot get both packages installed without problem.
Perhaps a solution would be to relax our dependency constraint to a lower version of matplotlib. Perhaps if we change to matplotlib>=2.2.3 that might do the trick?
@alanakbik That should work, but we probably should test the plotter/visualization functions with an older `matplotlib` version :D

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -16,7 +16,7 @@
         'gensim>=3.4.0',
         'tqdm>=4.26.0',
         'segtok>=1.5.7',
-        'matplotlib>=3.0.0',
+        'matplotlib>=2.2.3',
         'mpld3>=0.3',
         'sklearn',
         'sqlitedict>=1.6.0',


Test patch:
Repository: flairNLP/flair

Issue:
__version__ attribute?
I'm always frustrated when flair doesn't have a __version__attribute... :-)

Please, add a __version__attribute to the module.

Thank you!
DC



Discussion:

Resolving patch:
diff --git a/flair/__init__.py b/flair/__init__.py
--- a/flair/__init__.py
+++ b/flair/__init__.py
@@ -7,6 +7,7 @@
 
 import logging.config
 
+__version__ = "0.4.1"
 
 logging.config.dictConfig({
     'version': 1,


Test patch:
Repository: flairNLP/flair

Issue:
Error when training a model on CUDA
Hi,

I think this is a regression bug that was introduced after merging the CUDA semantics pull request #402.

Here's an error message that comes from the `test_model_integration.py`  integration test:

```bash
>               tag_tensor = torch.LongTensor(sentence_tags)
E               TypeError: expected torch.LongTensor (got torch.cuda.LongTensor)
```

I got a similar error message when I tried to test my recent ELMo transformer embeddings on GPU.

To replicate the error just run the integration tests (`pytest --runintegration`) on a machine with GPU/CUDA support :)




Discussion:

Resolving patch:
diff --git a/flair/models/sequence_tagger_model.py b/flair/models/sequence_tagger_model.py
--- a/flair/models/sequence_tagger_model.py
+++ b/flair/models/sequence_tagger_model.py
@@ -429,10 +429,7 @@ def _calculate_loss(self, features, lengths, tags) -> float:
             for sentence_feats, sentence_tags, sentence_length in zip(features, tags, lengths):
                 sentence_feats = sentence_feats[:sentence_length]
 
-                tag_tensor = torch.LongTensor(sentence_tags)
-                tag_tensor = tag_tensor.to(flair.device)
-
-                score += torch.nn.functional.cross_entropy(sentence_feats, tag_tensor)
+                score += torch.nn.functional.cross_entropy(sentence_feats, sentence_tags)
 
             return score
 


Test patch:
Repository: flairNLP/flair

Issue:
ELMO pubmed model
Elmo now has a new model trained on the PubMed corpus, check ( https://allennlp.org/elmo under contributed model)
Is this supported in Flair the same way the others (original, medium, small and Portuguese) are supported?
And is there a possibility to embed sentences using my own pre-trained ELMO/BERT models?
Thank you!


Discussion:
@nstfk The portugese model is currently supported.

If you want to use the new *PubMed* model, I'll open a PR to support it :)
Wow, that was quick 
And yes please, I am trying to evaluate and compare Flair/ELMO/BERT embedding on medical downstream tasks..

Resolving patch:
diff --git a/flair/embeddings.py b/flair/embeddings.py
--- a/flair/embeddings.py
+++ b/flair/embeddings.py
@@ -333,6 +333,9 @@ def __init__(self, model: str = 'original'):
         if model == 'pt' or model == 'portuguese':
             options_file = 'https://s3-us-west-2.amazonaws.com/allennlp/models/elmo/contributed/pt/elmo_pt_options.json'
             weight_file = 'https://s3-us-west-2.amazonaws.com/allennlp/models/elmo/contributed/pt/elmo_pt_weights.hdf5'
+        if model == 'pubmed':
+            options_file = 'https://s3-us-west-2.amazonaws.com/allennlp/models/elmo/contributed/pubmed/elmo_2x4096_512_2048cnn_2xhighway_options.json'
+            weight_file = 'https://s3-us-west-2.amazonaws.com/allennlp/models/elmo/contributed/pubmed/elmo_2x4096_512_2048cnn_2xhighway_weights_PubMed_only.hdf5'
 
         # put on Cuda if available
         from flair import device
@@ -508,7 +511,7 @@ def _add_embeddings_internal(self, sentences: List[Sentence]):
             longest_token_in_sentence = max(chars2_length)
             tokens_mask = torch.zeros((len(tokens_sorted_by_length), longest_token_in_sentence),
                                       dtype=torch.long, device=flair.device)
-            
+
             for i, c in enumerate(tokens_sorted_by_length):
                 tokens_mask[i, :chars2_length[i]] = torch.tensor(c, dtype=torch.long, device=flair.device)
 


Test patch:
Repository: flairNLP/flair

Issue:
BytePairEmbeddings serialization error
In the current master branch, the `BytePairEmbeddings` serialization creates problems when serialized on another machine than where they get deserialized.  

Specifically, this line causes the problem: 

https://github.com/zalandoresearch/flair/blob/4b3562eed534e4e305a36d2f22d12961bc6c25d5/flair/embeddings.py#L261 

Since this is only a path, this only works if the sentence piece model is already on your local machine under exactly the same path. If I train remotely and then deserialize locally, I get a `No such file or directory Error`.



Discussion:

Resolving patch:
diff --git a/flair/embeddings.py b/flair/embeddings.py
--- a/flair/embeddings.py
+++ b/flair/embeddings.py
@@ -1,3 +1,4 @@
+import os
 import re
 import logging
 from abc import abstractmethod
@@ -253,14 +254,32 @@ class BPEmbSerializable(BPEmb):
 
     def __getstate__(self):
         state = self.__dict__.copy()
+        # save the sentence piece model as binary file (not as path which may change)
+        state['spm_model_binary'] = open(self.model_file, mode='rb').read()
         state['spm'] = None
         return state
 
     def __setstate__(self, state):
         from bpemb.util import sentencepiece_load
-        state['spm'] = sentencepiece_load(state['model_file'])
+        model_file = self.model_tpl.format(lang=state['lang'], vs=state['vs'])
         self.__dict__ = state
 
+        # write out the binary sentence piece model into the expected directory
+        self.cache_dir: Path = Path(flair.file_utils.CACHE_ROOT) / 'embeddings'
+        if 'spm_model_binary' in self.__dict__:
+            # if the model was saved as binary and it is not found on disk, write to appropriate path
+            if not os.path.exists(self.cache_dir / state['lang']):
+                os.makedirs(self.cache_dir / state['lang'])
+            self.model_file = self.cache_dir / model_file
+            with open(self.model_file, 'wb') as out:
+                out.write(self.__dict__['spm_model_binary'])
+        else:
+            # otherwise, use normal process and potentially trigger another download
+            self.model_file = self._load_file(model_file)
+
+        # once the modes if there, load it with sentence piece
+        state['spm'] = sentencepiece_load(self.model_file)
+
 
 class BytePairEmbeddings(TokenEmbeddings):
 


Test patch:
Repository: flairNLP/flair

Issue:
Japanese FlairEmbeddings
Following the discussion in #308, @minh-agent and team were so kind as to provide their trained language models for Japanese. 

**Task**: Add Japanese language models so that they can be used in FlairEmbeddings.


Discussion:

Resolving patch:
diff --git a/flair/embeddings.py b/flair/embeddings.py
--- a/flair/embeddings.py
+++ b/flair/embeddings.py
@@ -759,6 +759,15 @@ def __init__(self, model: str, use_cache: bool = False, cache_directory: Path =
             base_path = 'https://s3.eu-central-1.amazonaws.com/alan-nlp/resources/embeddings-v0.4.1/pubmed-2015-bw-lm.pt'
             model = cached_path(base_path, cache_dir=cache_dir)
 
+        # Japanese forward
+        elif model.lower() == 'japanese-forward' or model.lower() == 'ja-forward':
+            base_path = 'https://s3.eu-central-1.amazonaws.com/alan-nlp/resources/embeddings-v0.4.1/lm__char-forward__ja-wikipedia-3GB/japanese-forward.pt'
+            model = cached_path(base_path, cache_dir=cache_dir)
+        # Japanese backward
+        elif model.lower() == 'japanese-backward' or model.lower() == 'ja-forward':
+            base_path = 'https://s3.eu-central-1.amazonaws.com/alan-nlp/resources/embeddings-v0.4.1/lm__char-backward__ja-wikipedia-3GB/japanese-backward.pt'
+            model = cached_path(base_path, cache_dir=cache_dir)
+
         elif not Path(model).exists():
             raise ValueError(f'The given model "{model}" is not available or is not a valid path.')
 


Test patch:
Repository: flairNLP/flair

Issue:
When running the calculate perplexity, got the error
Here is the error,  sounds likely it needs GPU, would you suggestion what I should I modify in the code of perplexity calculation?
File "/flair/models/language_model.py", line 69, in forward
    encoded = self.encoder(input)
  File ".local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File ".local/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 110, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File ".local/lib/python3.6/site-packages/torch/nn/functional.py", line 1110, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected object of type torch.cuda.LongTensor but found type torch.LongTensor for argument #3 'index'


Discussion:
Thanks for reporting it! I've found a fix for that + will open a PR soon :)

Resolving patch:
diff --git a/flair/models/language_model.py b/flair/models/language_model.py
--- a/flair/models/language_model.py
+++ b/flair/models/language_model.py
@@ -306,6 +306,7 @@ def calculate_perplexity(self, text: str) -> float:
 
         # input ids
         input = torch.tensor([self.dictionary.get_idx_for_item(char) for char in text[:-1]]).unsqueeze(1)
+        input = input.to(flair.device)
 
         # push list of character IDs through model
         hidden = self.init_hidden(1)
@@ -313,6 +314,7 @@ def calculate_perplexity(self, text: str) -> float:
 
         # the target is always the next character
         targets = torch.tensor([self.dictionary.get_idx_for_item(char) for char in text[1:]])
+        targets = targets.to(flair.device)
 
         # use cross entropy loss to compare output of forward pass with targets
         cross_entroy_loss = torch.nn.CrossEntropyLoss()


Test patch:
Repository: flairNLP/flair

Issue:
Release 0.4.1: NameError: name 'bert_model' is not defined
Hi,

I made an error during resolving a merge conflict for PR #523.

`BertEmbeddings` are currently not working, because of:

```bash
Traceback (most recent call last):
  File "ner_en_bert.py", line 17, in <module>
    BertEmbeddings()
  File "/mnt/flair/flair/embeddings.py", line 1070, in __init__
    if bert_model not in BERT_PRETRAINED_MODEL_ARCHIVE_MAP.keys():
NameError: name 'bert_model' is not defined
```

I'm going to fix that!


Discussion:

Resolving patch:
diff --git a/flair/embeddings.py b/flair/embeddings.py
--- a/flair/embeddings.py
+++ b/flair/embeddings.py
@@ -1067,7 +1067,7 @@ def __init__(self,
         """
         super().__init__()
 
-        if bert_model not in BERT_PRETRAINED_MODEL_ARCHIVE_MAP.keys():
+        if bert_model_or_path not in BERT_PRETRAINED_MODEL_ARCHIVE_MAP.keys():
             raise ValueError('Provided bert-model is not available.')
 
         self.tokenizer = BertTokenizer.from_pretrained(bert_model_or_path)


Test patch:
Repository: flairNLP/flair

Issue:
[BUG] A clear Exception/Error message must be thrown while trying to create a Sentence object from an empty string
**Describe the bug**
If one tries to create a `Sentence` object from an empty string, an `UnboundLocalError` of the `index` variable is thrown

**To Reproduce**
```python3
>>> from flair.data import Sentence
>>> s = ''
>>> Sentence(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\XYZ\AppData\Local\Programs\Python\Python36\lib\site-packages\flair\data.py", line 338, in __init__
    index += 1
UnboundLocalError: local variable 'index' referenced before assignment
```

**Expected behavior**
Something like `ValueError: Cannot create a Sentence object from an empty string` or some `Error` or `Exception` mentioning that the string is empty or a `Sentence object "" - 0 tokens`

**Environment (please complete the following information):**
   I feel this is applicable to any OS/version, but anyway:
 - Windows 10
 - flair-0.4.1
 - Python 3.6.8



Discussion:

Resolving patch:
diff --git a/flair/data.py b/flair/data.py
--- a/flair/data.py
+++ b/flair/data.py
@@ -323,6 +323,9 @@ def __init__(self, text: str = None, use_tokenizer: bool = False, labels: Union[
 
             # otherwise assumes whitespace tokenized text
             else:
+                # catch the empty string case
+                if not text:
+                    raise ValueError("Cannot convert empty string to a Sentence object.")
                 # add each word in tokenized string as Token object to Sentence
                 word = ''
                 for index, char in enumerate(text):


Test patch:
diff --git a/tests/test_data.py b/tests/test_data.py
--- a/tests/test_data.py
+++ b/tests/test_data.py
@@ -22,6 +22,13 @@ def test_get_head():
     assert (token1 == token2.get_head())
     assert (None == token1.get_head())
 
+def test_create_sentence_on_empty_string():
+
+    with pytest.raises(ValueError) as e:
+        sentence: Sentence = Sentence('')
+
+        assert (e.type is ValueError)
+        assert (e.value.args[0] == "Cannot convert empty string to a Sentence object.")
 
 def test_create_sentence_without_tokenizer():
     sentence: Sentence = Sentence('I love Berlin.')
Repository: flairNLP/flair

Issue:
one redundant code line
in TextClassifier class
`
34 self.document_embeddings = document_embeddings
`
`
38 self.document_embeddings: flair.embeddings.DocumentLSTMEmbeddings = document_embeddings
`


Discussion:

Resolving patch:
diff --git a/flair/models/text_classification_model.py b/flair/models/text_classification_model.py
--- a/flair/models/text_classification_model.py
+++ b/flair/models/text_classification_model.py
@@ -31,12 +31,10 @@ def __init__(self,
 
         super(TextClassifier, self).__init__()
 
-        self.document_embeddings = document_embeddings
+        self.document_embeddings: flair.embeddings.DocumentLSTMEmbeddings = document_embeddings
         self.label_dictionary: Dictionary = label_dictionary
         self.multi_label = multi_label
 
-        self.document_embeddings: flair.embeddings.DocumentLSTMEmbeddings = document_embeddings
-
         self.decoder = nn.Linear(self.document_embeddings.embedding_length, len(self.label_dictionary))
 
         self._init_weights()


Test patch:
Repository: flairNLP/flair

Issue:
PooledFlairEmbeddings:  expected type torch.cuda.FloatTensor but got torch.FloatTensor
Hi,

when loading a trained model with PooledFlairEmbeddings (on GPU), the following error message
is thrown:

```bash
raceback (most recent call last):
  File "predict.py", line 31, in <module>
    tagger.predict(tagged_sentence)
  File "/mnt/flair/flair/models/sequence_tagger_model.py", line 300, in predict
    tags, _ = self.forward_labels_and_loss(batch, sort=False)
  File "/mnt/flair/flair/models/sequence_tagger_model.py", line 268, in forward_labels_and_loss
    feature, lengths, tags = self.forward(sentences, sort=sort)
  File "/mnt/flair/flair/models/sequence_tagger_model.py", line 315, in forward
    self.embeddings.embed(sentences)
  File "/mnt/flair/flair/embeddings.py", line 130, in embed
    embedding.embed(sentences)
  File "/mnt/flair/flair/embeddings.py", line 63, in embed
    self._add_embeddings_internal(sentences)
  File "/mnt/flair/flair/embeddings.py", line 1031, in _add_embeddings_internal
    aggregated_embedding = self.aggregate_op(self.word_embeddings[token.text], local_embedding)
RuntimeError: expected type torch.cuda.FloatTensor but got torch.FloatTensor
```

Tested with latest `master` (cf56e8413cae79707545f5eddf3a88c05810d0d3)


Discussion:
Hello @stefan-it thanks for reporting this. Since I don't currently have good access to a GPU machine, could you try loading the pre-trained 'ner-multi-fast-learn' sequence tagger (`tagger = SequenceTagger.load('ner-multi-fast-learn')`) to see if the problem also applies here?
I can see no problems with that model :)

Meanwhile I think I found a patch for that issue:

```python
local_embedding = token._embeddings[self.context_embeddings.name]
local_embedding = local_embedding.to(flair.device)
```

The last line was missing - after adding it, the test mode runs fine after training.

Resolving patch:
diff --git a/flair/embeddings.py b/flair/embeddings.py
--- a/flair/embeddings.py
+++ b/flair/embeddings.py
@@ -1021,6 +1021,7 @@ def _add_embeddings_internal(self, sentences: List[Sentence]) -> List[Sentence]:
 
                 # update embedding
                 local_embedding = token._embeddings[self.context_embeddings.name]
+                local_embedding = local_embedding.to(flair.device)
 
                 if token.text[0].isupper() or not self.only_capitalized:
 


Test patch: