internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: horovod/horovod

Issue:
No module named 'fsspec.callbacks' thrown at horovod/spark/common/store.py ln 33
**Environment:**
1. Framework: TensorFlow
2. Framework version: 2.6.2
3. Horovod version: 0.24.1
4. MPI version: 4.1.0
5. CUDA version: N/A
6. NCCL version: N/A
7. Python version: 3.7
8. Spark / PySpark version: 3.2
9. Ray version: N/A
10. OS and version: Ubuntu 18.04
11. GCC version: 9.3.1
12. CMake version: 2.8

**Checklist:** >>>>>>>>>>>>>>>>> all "YES"
1. Did you search issues to find if somebody asked this question before?
2. If your question is about hang, did you read [this doc](https://github.com/horovod/horovod/blob/master/docs/running.rst)?
3. If your question is about docker, did you read [this doc](https://github.com/horovod/horovod/blob/master/docs/docker.rst)?
4. Did you check if you question is answered in the [troubleshooting guide](https://github.com/horovod/horovod/blob/master/docs/troubleshooting.rst)?

**Bug report:**
Please describe erroneous behavior you're observing and steps to reproduce it.
1. Create a GCP Dataproc cluster with 2.0.27-ubuntu18 image
2. install TF 2.6.2 and Horovod 0.24.1
3. `>> from horovod.spark.common.store import HDFSStore`
output:
Traceback (most recent call last):
  File "/opt/conda/default/lib/python3.8/site-packages/horovod/spark/common/store.py", line 33, in <module>
    from fsspec.callbacks import _DEFAULModuleNotFoundError: No module named 'fsspec.callbacks'

The fsspec.callback module was introduced in https://github.com/fsspec/filesystem_spec/releases/tag/2021.07.0

The line
https://github.com/horovod/horovod/blob/ebd135098571722469bb6290a6d098a9e1c96574/setup.py#L169
should be
`spark_require_list = ['numpy', 'petastorm>=0.11.0', 'pyarrow>=0.15.0', 'fsspec>=2021.07.0']`



Discussion:
fsspec.callback commit id

https://github.com/fsspec/filesystem_spec/commit/47f72bcaa34d46255d6675f8db12b2db46d5699f
@kamalsharma2 FYI
@zyluo Would you like to create a PR? Thanks for finding it out.
@chongxiaoc It would take quite a while for me to submit a PR. I'll wait for @kamalsharma2 to create one.
> @chongxiaoc It would take quite a while for me to submit a PR. I'll wait for @kamalsharma2 to create one.

I will create a one for you then.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -166,7 +166,7 @@ def build_extensions(self):
 mxnet_require_list = ['mxnet>=1.4.1']
 pyspark_require_list = ['pyspark>=2.3.2;python_version<"3.8"',
                         'pyspark>=3.0.0;python_version>="3.8"']
-spark_require_list = ['numpy', 'petastorm>=0.11.0', 'pyarrow>=0.15.0', 'fsspec']
+spark_require_list = ['numpy', 'petastorm>=0.11.0', 'pyarrow>=0.15.0', 'fsspec>=2021.07.0']
 # https://github.com/ray-project/ray/pull/17465
 ray_require_list = ['ray', 'aioredis<2']
 pytorch_spark_require_list = pytorch_require_list + \


Test patch:
Repository: horovod/horovod

Issue:
No module named 'packaging' when install horovod
It seems that the horovod v0.26.0 has some dependency problems.

How long does it take for a new patch version to be released or should I pin the horovod version? ^_^

![image](https://user-images.githubusercontent.com/32220263/195746141-2a0050f5-1eaf-4f7b-9a62-50fd6b13f2ff.png)



Discussion:
ping @tgaddair have a look ~

Resolving patch:
diff --git a/horovod/__init__.py b/horovod/__init__.py
--- a/horovod/__init__.py
+++ b/horovod/__init__.py
@@ -1,3 +1,3 @@
 from horovod.runner import run
 
-__version__ = '0.26.0'
+__version__ = '0.26.1'


Test patch:
Repository: horovod/horovod

Issue:
Spark tests fail with AttributeError: 'pyarrow.lib.Schema' object has no attribute 'to_arrow_schema'
Spark tests fail with

```
Traceback (most recent call last):
  File "/horovod/examples/spark/pytorch/pytorch_spark_mnist.py", line 122, in <module>
    torch_model = torch_estimator.fit(train_df).setOutputCols(['label_prob'])
  File "/usr/local/lib/python3.8/dist-packages/horovod/spark/common/estimator.py", line 35, in fit
    return super(HorovodEstimator, self).fit(df, params)
  File "/usr/local/lib/python3.8/dist-packages/pyspark/ml/base.py", line 205, in fit
    return self._fit(dataset)
  File "/usr/local/lib/python3.8/dist-packages/horovod/spark/common/estimator.py", line 68, in _fit
    with util.prepare_data(backend.num_processes(),
  File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.8/dist-packages/horovod/spark/common/util.py", line 735, in prepare_data
    dataset_idx = _get_or_create_dataset(key, store, df, feature_columns, label_columns,
  File "/usr/local/lib/python3.8/dist-packages/horovod/spark/common/util.py", line 672, in _get_or_create_dataset
    train_rows, val_rows, pq_metadata, avg_row_size = get_simple_meta_from_parquet(
  File "/usr/local/lib/python3.8/dist-packages/horovod/spark/common/util.py", line 495, in get_simple_meta_from_parquet
    train_data_schema = train_data.schema.to_arrow_schema()
AttributeError: 'pyarrow.lib.Schema' object has no attribute 'to_arrow_schema'
```
https://github.com/horovod/horovod/actions/runs/4025636527/jobs/6919132641

It's probably related to the 11.0 release of pyarrow. https://pypi.org/project/pyarrow/#history


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -167,7 +167,7 @@ def build_extensions(self):
 mxnet_require_list = ['mxnet>=1.4.1']
 pyspark_require_list = ['pyspark>=2.3.2;python_version<"3.8"',
                         'pyspark>=3.0.0;python_version>="3.8"']
-spark_require_list = ['numpy', 'petastorm>=0.12.0', 'pyarrow>=0.15.0', 'fsspec>=2021.07.0']
+spark_require_list = ['numpy', 'petastorm>=0.12.0', 'pyarrow>=0.15.0,<11.0', 'fsspec>=2021.07.0']
 # https://github.com/ray-project/ray/pull/17465
 # google-api-core>=2.9.0 depends on protobuf<5.0.0dev,>=3.20.1, which conflicts with
 #   tensorflow protobuf~=3.20 and pytorch-lightning protobuf<3.20,>=3.9.2


Test patch:
Repository: flairNLP/flair

Issue:
getting items in sentence indices 
2 issues here

- the ```__getitem__``` should return something for sentence[0] (in python this is usually the first thing)
- the ```flair.data.Token``` object should display the string in it's __repr__ method


Discussion:

Resolving patch:
diff --git a/flair/data.py b/flair/data.py
--- a/flair/data.py
+++ b/flair/data.py
@@ -124,6 +124,9 @@ def get_head(self):
     def __str__(self) -> str:
         return 'Token: %d %s' % (self.idx, self.text)
 
+    def __repr__(self) -> str:
+        return 'Token: %d %s' % (self.idx, self.text)
+
     def set_embedding(self, name: str, vector: torch.autograd.Variable):
         self._embeddings[name] = vector.cpu()
 
@@ -174,8 +177,8 @@ def __init__(self, text: str = None, use_tokenizer: bool = False, labels: List[s
             for word in text.split(' '):
                 self.add_token(Token(word))
 
-    def __getitem__(self, token_id: int) -> Token:
-        return self.get_token(token_id)
+    def __getitem__(self, idx: int) -> Token:
+        return self.tokens[idx]
 
     def __iter__(self):
         return iter(self.tokens)
@@ -194,7 +197,8 @@ def add_labels(self, labels: List[str]):
 
     def get_token(self, token_id: int) -> Token:
         for token in self.tokens:
-            if token.idx == token_id: return token
+            if token.idx == token_id:
+                return token
 
     def add_token(self, token: Token):
         self.tokens.append(token)


Test patch:
diff --git a/tests/test_data.py b/tests/test_data.py
--- a/tests/test_data.py
+++ b/tests/test_data.py
@@ -1,5 +1,7 @@
 import os
 
+import pytest
+
 from flair.data import Sentence, Token, Dictionary, TaggedCorpus
 
 
@@ -43,6 +45,16 @@ def test_sentence_to_plain_string():
     assert ('I love Berlin .' == sentence.to_plain_string())
 
 
+def test_sentence_get_item():
+    sentence: Sentence = Sentence('I love Berlin.', use_tokenizer=True)
+
+    assert (sentence.get_token(1) == sentence[0])
+    assert (sentence.get_token(3) == sentence[2])
+
+    with pytest.raises(IndexError):
+        token = sentence[4]
+
+
 def test_sentence_to_tagged_string():
     token1 = Token('I', 0)
     token2 = Token('love', 1, 0)
Repository: flairNLP/flair

Issue:
Set `use_cache` to False when saving a model
When training a model using embeddings that are using the cache options (embeddings are written to disk during training to speed up training time) the cache option is saved along the model. When loading the model from disk again on a different machine to predict some data points, the cache files might not be there. This results in an error loading the model. 

Solution: Disable cache option once the model is saved.


Discussion:

Resolving patch:
diff --git a/flair/embeddings.py b/flair/embeddings.py
--- a/flair/embeddings.py
+++ b/flair/embeddings.py
@@ -452,6 +452,8 @@ def __getstate__(self):
         state = self.__dict__.copy()
         # Remove the unpicklable entries.
         state['cache'] = None
+        state['use_cache'] = False
+        state['cache_directory'] = None
         return state
 
     @property
@@ -460,8 +462,11 @@ def embedding_length(self) -> int:
 
     def _add_embeddings_internal(self, sentences: List[Sentence]) -> List[Sentence]:
 
+        cache_path = '{}-tmp-cache.sqllite'.format(self.name) if self.cache_directory is None else os.path.join(
+            self.cache_directory, '{}-tmp-cache.sqllite'.format(os.path.basename(self.name)))
+
         # by default, use_cache is false (for older pre-trained models TODO: remove in version 0.4)
-        if 'cache' not in self.__dict__:
+        if 'cache' not in self.__dict__ or 'cache_directory' not in self.__dict__ or not os.path.exists(cache_path):
             self.use_cache = False
             self.cache_directory = None
 
@@ -470,9 +475,6 @@ def _add_embeddings_internal(self, sentences: List[Sentence]) -> List[Sentence]:
 
             # lazy initialization of cache
             if not self.cache:
-                cache_path = '{}-tmp-cache.sqllite'.format(self.name) if self.cache_directory is None else os.path.join(
-                    self.cache_directory, '{}-tmp-cache.sqllite'.format(os.path.basename(self.name)))
-
                 from sqlitedict import SqliteDict
                 self.cache = SqliteDict(cache_path, autocommit=True)
 


Test patch:
Repository: flairNLP/flair

Issue:
Text Classifier: Clear embeddings in predict()
In the `predict()` method of the text classification model the sentence embeddings are not cleared after calculation. This might lead to memory issues. 

TODO: Clear embeddings at the end of the predict method.


Discussion:

Resolving patch:
diff --git a/flair/models/text_classification_model.py b/flair/models/text_classification_model.py
--- a/flair/models/text_classification_model.py
+++ b/flair/models/text_classification_model.py
@@ -99,7 +99,7 @@ def load_from_file(cls, model_file):
         model.eval()
         return model
 
-    def predict(self, sentences: Union[Sentence, List[Sentence]], mini_batch_size: int = 32, embeddings_in_memory: bool = True) -> List[Sentence]:
+    def predict(self, sentences: Union[Sentence, List[Sentence]], mini_batch_size: int = 32) -> List[Sentence]:
         """
         Predicts the class labels for the given sentences. The labels are directly added to the sentences.
         :param sentences: list of sentences
@@ -118,8 +118,7 @@ def predict(self, sentences: Union[Sentence, List[Sentence]], mini_batch_size: i
             for (sentence, labels) in zip(batch, predicted_labels):
                 sentence.labels = labels
 
-            if not embeddings_in_memory:
-                clear_embeddings(batch)
+            clear_embeddings(batch)
 
         return sentences
 


Test patch:
Repository: flairNLP/flair

Issue:
Flair depends on AWS CLI (boto library etc) --- should be removed/optional
I assume that the usual Flair user does not need any AWS dependency, therefore the hard AWS CLI dependency (defined in `setup.py`) should be removed or at least made optional.


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -13,14 +13,12 @@
     license='MIT',
     install_requires=[
         'torch==0.4.1',
-        'awscli==1.14.32',
         'gensim==3.4.0',
         'typing==3.6.4',
         'tqdm==4.23.4',
         'segtok==1.5.6',
         'matplotlib==3.0.0',
         'mpld3==0.3',
-        'jinja2==2.10',
         'sklearn',
         'sqlitedict==1.6.0',
     ],


Test patch:
Repository: flairNLP/flair

Issue:
Test data in language model
Is somewhere used test data?
https://github.com/zalandoresearch/flair/blob/d7c0f1733b398d59b3bffb62f5a422343c7e420e/flair/trainers/language_model_trainer.py#L35


Discussion:
No, after the training there was no evaluation on test set in the recent language model I trained :(
Oh yes this is a mistake - we need to add a final evaluation on test data after all epochs or on exit from the loop to the `LanguageModelTrainer`. Thanks for spotting this @djstrong and @stefan-it

@stefan-it how are the language models coming? :)
~~Two weeks ago training worked, but now (with new GPU + own compiled `pytorch` version) it is currently not working, see #180~~ Currently training :sunglasses: 

Resolving patch:
diff --git a/flair/trainers/language_model_trainer.py b/flair/trainers/language_model_trainer.py
--- a/flair/trainers/language_model_trainer.py
+++ b/flair/trainers/language_model_trainer.py
@@ -299,6 +299,19 @@ def train(self,
             log.info('-' * 89)
             log.info('Exiting from training early')
 
+        ###############################################################################
+        # final testing
+        ###############################################################################
+        test_data = self._batchify(self.corpus.test, mini_batch_size)
+        test_loss = self.evaluate(test_data, mini_batch_size, sequence_length)
+
+        summary = 'TEST: valid loss {:5.2f} | valid ppl {:8.2f}'.format(test_loss, math.exp(test_loss))
+        with open(loss_txt, "a") as myfile:
+            myfile.write('%s\n' % summary)
+
+        log.info(summary)
+        log.info('-' * 89)
+
     def evaluate(self, data_source, eval_batch_size, sequence_length):
         # Turn on evaluation mode which disables dropout.
         self.model.eval()


Test patch:
Repository: flairNLP/flair

Issue:
Add torch.no_grad() to LanguageModel.generate_text()
The autograd engine is not required when using an LM to generate text.

So, as pointed out in #167, `torch.no_grad()` needs to be added to `LanguageModel.generate_text()` for better performance and to avoid out of memory issues.


Discussion:

Resolving patch:
diff --git a/flair/models/language_model.py b/flair/models/language_model.py
--- a/flair/models/language_model.py
+++ b/flair/models/language_model.py
@@ -150,22 +150,23 @@ def save(self, file):
         torch.save(model_state, file, pickle_protocol=4)
 
     def generate_text(self, number_of_characters=1000) -> str:
-        characters = []
-
-        idx2item = self.dictionary.idx2item
-
-        # initial hidden state
-        hidden = self.init_hidden(1)
-        input = torch.rand(1, 1).mul(len(idx2item)).long()
-        if torch.cuda.is_available():
-            input = input.cuda()
-
-        for i in range(number_of_characters):
-            prediction, rnn_output, hidden = self.forward(input, hidden)
-            word_weights = prediction.squeeze().data.div(1.0).exp().cpu()
-            word_idx = torch.multinomial(word_weights, 1)[0]
-            input.data.fill_(word_idx)
-            word = idx2item[word_idx].decode('UTF-8')
-            characters.append(word)
-
-        return ''.join(characters)
+        with torch.no_grad():
+            characters = []
+
+            idx2item = self.dictionary.idx2item
+
+            # initial hidden state
+            hidden = self.init_hidden(1)
+            input = torch.rand(1, 1).mul(len(idx2item)).long()
+            if torch.cuda.is_available():
+                input = input.cuda()
+
+            for i in range(number_of_characters):
+                prediction, rnn_output, hidden = self.forward(input, hidden)
+                word_weights = prediction.squeeze().data.div(1.0).exp().cpu()
+                word_idx = torch.multinomial(word_weights, 1)[0]
+                input.data.fill_(word_idx)
+                word = idx2item[word_idx].decode('UTF-8')
+                characters.append(word)
+
+            return ''.join(characters)


Test patch:
diff --git a/tests/test_model_integration.py b/tests/test_model_integration.py
--- a/tests/test_model_integration.py
+++ b/tests/test_model_integration.py
@@ -298,5 +298,9 @@ def test_train_language_model(results_base_path, resources_path):
     sentence = Sentence('I love Berlin')
     char_lm_embeddings.embed(sentence)
 
+    text = language_model.generate_text(100)
+    assert (text is not None)
+    assert (len(text) == 100)
+
     # clean up results directory
     shutil.rmtree(results_base_path, ignore_errors=True)
Repository: flairNLP/flair

Issue:
Language model training
Hi,

with the latest `0.4` branch, training of a language model does not work. More precisely, the `TextCorpus` class expects a `Path`-like object as training path (see [here](https://github.com/zalandoresearch/flair/blob/release-0.4/flair/trainers/language_model_trainer.py#L19)). With a `Path`-like object the following comprehension won't work:

```python
self.train_files = sorted(
    [f for f in self.train_path.iterdir() if (self.train_path / f).exists()])
```

-> `(self.train_path / f).exists()` will always be empty, so the list of training files will be empty. I could temporarily fix it with using only `f.exists()` but then another error appeared.

It just want to mention this issue before the official `0.4` version is released :)


Discussion:
Thanks for spotting this!

Resolving patch:
diff --git a/flair/trainers/language_model_trainer.py b/flair/trainers/language_model_trainer.py
--- a/flair/trainers/language_model_trainer.py
+++ b/flair/trainers/language_model_trainer.py
@@ -23,7 +23,7 @@ def __init__(self, path: Path, dictionary: Dictionary, forward: bool = True, cha
         self.train_path = path / 'train'
 
         self.train_files = sorted(
-            [f for f in self.train_path.iterdir() if (self.train_path / f).exists()])
+            [f for f in self.train_path.iterdir() if f.exists()])
 
         self.dictionary: Dictionary = dictionary
 
@@ -53,7 +53,7 @@ def get_next_train_slice(self):
 
         current_train_file = self.train_files[self.current_train_file_index]
 
-        train_slice = self.charsplit(self.train_path / current_train_file,
+        train_slice = self.charsplit(current_train_file,
                                     expand_vocab=False,
                                     forward=self.forward,
                                     split_on_char=self.split_on_char)


Test patch:
diff --git a/tests/test_language_model.py b/tests/test_language_model.py
new file mode 100644
--- /dev/null
+++ b/tests/test_language_model.py
@@ -0,0 +1,20 @@
+import shutil
+
+from flair.data import Dictionary
+from flair.trainers.language_model_trainer import TextCorpus
+
+
+def test_train_resume_language_model_training(resources_path, results_base_path, tasks_base_path):
+    # get default dictionary
+    dictionary: Dictionary = Dictionary.load('chars')
+
+    # get the example corpus and process at character level in forward direction
+    corpus: TextCorpus = TextCorpus(resources_path / 'corpora/lorem_ipsum',
+                                dictionary,
+                                forward=True,
+                                character_level=True)
+
+    assert (corpus.test is not None)
+    assert (corpus.train_files is not None)
+    assert (corpus.valid is not None)
+    assert (len(corpus.train_files) == 2)
Repository: flairNLP/flair

Issue:
Update torch version to 1.0.0
torch version 1.0.0 is available. Use torch version 1.0.0 in Flair.


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -12,7 +12,7 @@
     packages=find_packages(exclude='test'),  # same as name
     license='MIT',
     install_requires=[
-        'torch==0.4.1',
+        'torch==1.0.0',
         'gensim==3.4.0',
         'typing==3.6.4',
         'tqdm==4.26.0',


Test patch:
Repository: flairNLP/flair

Issue:
macro_avg_f_score bug
In the file flair/training_utils.py, when calculating macro_avg_f_score, I think 'recall' method should be called but 'precision' is called.

![image](https://user-images.githubusercontent.com/44130753/51589926-4b59bf00-1f23-11e9-964a-24ac0bc82589.png)

i think it needs to write like this:
`def macro_avg_f_score(self): 
         class_precisions = [self.precision(class_name) for class_name in self.get_classes()] 
         class_recalls = [self.recall(class_name) for class_name in self.get_classes()]`




Discussion:
Thanks for spotting this!

Resolving patch:
diff --git a/flair/training_utils.py b/flair/training_utils.py
--- a/flair/training_utils.py
+++ b/flair/training_utils.py
@@ -80,7 +80,7 @@ def micro_avg_f_score(self):
 
     def macro_avg_f_score(self):
         class_precisions = [self.precision(class_name) for class_name in self.get_classes()]
-        class_recalls = [self.precision(class_name) for class_name in self.get_classes()]
+        class_recalls = [self.recall(class_name) for class_name in self.get_classes()]
 
         if len(class_precisions) > 0 and len(class_recalls) > 0:
             macro_precision = sum(class_precisions) / len(class_precisions)


Test patch:
diff --git a/tests/test_utils.py b/tests/test_utils.py
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -63,7 +63,7 @@ def test_metric_with_classes():
     assert(metric.accuracy('class-4') == 0.75)
 
     assert(metric.micro_avg_f_score() == 0.2184)
-    assert(metric.macro_avg_f_score() == 0.4)
+    assert(metric.macro_avg_f_score() == 0.5714)
 
     assert(metric.micro_avg_accuracy() == 0.1696)
     assert(metric.macro_avg_accuracy() == 0.5875)