Repository: flairNLP/flair

Issue:
encoding error while loading WIKINER_RUSSIAN
I am trying to load russian NER labelled data 'WIKINER_RUSSIAN' into Flair.
But getting some encoding error during loading.
My System spec:
Flair: 0.4.3
Python: 3.8.0

![image](https://user-images.githubusercontent.com/13539449/69625081-dc318780-106b-11ea-8060-cf841ae795d4.png)



Discussion:
Hi @santoshboina ,

could you try to use the latest version from `master` 🤔

I got it running with:

```python
In [2]: ru_corpus = flair.datasets.WIKINER_RUSSIAN()                                                                                                                                                                                                                                                          
2019-11-26 12:20:11,585 https://raw.githubusercontent.com/dice-group/FOX/master/input/Wikiner/aij-wikiner-ru-wp3.bz2 not found in cache, downloading to /tmp/tmpsxh4xad0
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7856559/7856559 [00:00<00:00, 54409426.05B/s]
2019-11-26 12:20:11,778 copying /tmp/tmpsxh4xad0 to cache at /home/stefan/.flair/datasets/wikiner_russian/aij-wikiner-ru-wp3.bz2
2019-11-26 12:20:11,784 removing temp file /tmp/tmpsxh4xad0
2019-11-26 12:20:16,692 Reading data from /home/stefan/.flair/datasets/wikiner_russian
2019-11-26 12:20:16,692 Train: /home/stefan/.flair/datasets/wikiner_russian/aij-wikiner-ru-wp3.train
2019-11-26 12:20:16,692 Dev: None
2019-11-26 12:20:16,692 Test: None
```

If that's not working, I guess it has something to do with encoding on Windows. Maybe we need to add utf-8 explict. as input encoding in the dataset reader...

I have tried installing latest version but I am follwing issue:
![image](https://user-images.githubusercontent.com/13539449/69626929-a1315300-106f-11ea-8349-4944291d15d9.png)

I think you need to update your PyTorch version to at least 1.2 (related to this [issue](https://github.com/zalandoresearch/flair/issues/1236)) :)
Yes ,using PyTorch 1.3 and Flair 0.4.4. But still I am facing encoding issue while loading WIKINER_RUSSIAN dataset
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stefan-it 
i got the same error on windows.
flair: 0.5
torch: 1.5.0+cpu

yes, adding explicit encoding in `_download_wikiner` fixed this issue
```python
with bz_file as f, open(
....
encoding="utf-8"
)
```

Resolving patch:
diff --git a/flair/datasets/sequence_labeling.py b/flair/datasets/sequence_labeling.py
--- a/flair/datasets/sequence_labeling.py
+++ b/flair/datasets/sequence_labeling.py
@@ -1310,6 +1310,7 @@ def _download_wikiner(language_code: str, dataset_name: str):
                 / dataset_name
                 / f"aij-wikiner-{lc}-wp3.train",
                 "w",
+                encoding="utf-8"
         ) as out:
             for line in f:
                 line = line.decode("utf-8")


Test patch: