rIsHu009/github-issues-updated huggingface dataset:
Title: OverflowError when slicing with an array containing negative ids
Author: lhoestq (MEMBER)
State: closed
Created: 2020-09-24T16:27:14Z

```python
from datasets import Dataset

d = ds.Dataset.from_dict({"a": range(10)})

print(d[0])
# {'a': 0}

print(d[-1])
# {'a': 9}

print(d[[0, -1]])
# OverflowError
```
results in
```
---------------------------------------------------------------------------
OverflowError                             Traceback (most recent call last)
<ipython-input-5-863dc3555598> in <module>
----> 1 d[[0, -1]]

~/Desktop/hf/nlp/src/datasets/arrow_dataset.py in __getitem__(self, key)
   1070             format_columns=self._format_columns,
   1071             output_all_columns=self._output_all_columns,
-> 1072             format_kwargs=self._format_kwargs,
   1073         )
   1074 

~/Desktop/hf/nlp/src/datasets/arrow_dataset.py in _getitem(self, key, format_type, format_columns, output_all_columns, format_kwargs)
   1025                 indices = key
   1026 
-> 1027             indices_array = pa.array([int(i) for i in indices], type=pa.uint64())
   1028 
   1029             # Check if we need to convert indices

~/.virtualenvs/hf-datasets/lib/python3.7/site-packages/pyarrow/array.pxi in pyarrow.lib.array()

~/.virtualenvs/hf-datasets/lib/python3.7/site-packages/pyarrow/array.pxi in pyarrow.lib._sequence_to_array()

OverflowError: can't convert negative value to unsigned int
```

Comments:
Title: Loss not decrease with Datasets and Transformers
Author: wangcongcong123 (NONE)
State: closed
Created: 2020-09-24T15:14:43Z

HI,

The following script is used to fine-tune a BertForSequenceClassification model on SST2.

The script is adapted from [this colab](https://colab.research.google.com/github/huggingface/datasets/blob/master/notebooks/Overview.ipynb) that presents an example of fine-tuning BertForQuestionAnswering using squad dataset. In that colab, loss works fine. When I adapt it to SST2, the loss fails to decrease as it should. I attach the adapted script below and appreciate anyone pointing out what I miss?

```python
import torch
from datasets import load_dataset
from transformers import BertForSequenceClassification
from transformers import BertTokenizerFast
# Load our training dataset and tokenizer
dataset = load_dataset("glue", 'sst2')
tokenizer = BertTokenizerFast.from_pretrained('bert-base-cased')
del dataset["test"] # let's remove it in this demo

# Tokenize our training dataset
def convert_to_features(example_batch):
    encodings = tokenizer(example_batch["sentence"])
    encodings.update({"labels": example_batch["label"]})
    return encodings

encoded_dataset = dataset.map(convert_to_features, batched=True)
# Format our dataset to outputs torch.Tensor to train a pytorch model
columns = ['input_ids', 'token_type_ids', 'attention_mask', 'labels']
encoded_dataset.set_format(type='torch', columns=columns)

# Instantiate a PyTorch Dataloader around our dataset
# Let's do dynamic batching (pad on the fly with our own collate_fn)
def collate_fn(examples):
    return tokenizer.pad(examples, return_tensors='pt')

dataloader = torch.utils.data.DataLoader(encoded_dataset['train'], collate_fn=collate_fn, batch_size=8)
# Now let's train our model
device = 'cuda' if torch.cuda.is_available() else 'cpu'
# Let's load a pretrained Bert model and a simple optimizer
model = BertForSequenceClassification.from_pretrained('bert-base-cased', return_dict=True)
optimizer = torch.optim.Adam(model.parameters(), lr=1e-5)
model.train().to(device)
for i, batch in enumerate(dataloader):
    batch.to(device)
    outputs = model(**batch)
    loss = outputs.loss
    loss.backward()
    optimizer.step()
    model.zero_grad()
    print(f'Step {i} - loss: {loss:.3}')


```
In case needed.

- datasets == 1.0.2
- transformers == 3.2.0

Comments:
And I tested it on T5ForConditionalGeneration, that works no problem.

Hi did you manage to fix your issue ?

If so feel free to share your fix and close this thread
Title: load_dataset from local squad.py, raise error: TypeError: 'NoneType' object is not callable 
Author: xixiaoyao (NONE)
State: closed
Created: 2020-09-23T03:53:36Z

version: 1.0.2

```
train_dataset  = datasets.load_dataset('squad') 
```

The above code can works. However, when I download the squad.py from your server, and saved as `my_squad.py` to local. I run followings raise errors.
```
train_dataset  = datasets.load_dataset('./my_squad.py')                                                                                                
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-28-25a84b4d1581> in <module>
----> 1 train_dataset  = nlp.load_dataset('./my_squad.py')

/opt/conda/lib/python3.7/site-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, save_infos, script_version, **config_kwargs)
    602         hash=hash,
    603         features=features,
--> 604         **config_kwargs,
    605     )
    606 

TypeError: 'NoneType' object is not callable

Comments:
Hi !
Thanks for reporting.
It looks like no object inherits from `datasets.GeneratorBasedBuilder` (or more generally from `datasets.DatasetBuilder`) in your script.

Could you check that there exist at least one dataset builder class ?

Hi @xixiaoyao did you manage to fix your issue ?

No activity, closing

It happened when try to change the old project which use 'nlp' to new project which use 'datasets'. You should check you old 'my_squad.py' file, change the inherit class from `nlp.xxx` to `datasets.xxx`. Otherwise datasets - load.py - import_main_class() `if inspect.isclass(obj) and issubclass(obj, main_cls_type):` can not find the main_cls.
Title: Keep new columns in transmit format
Author: lhoestq (MEMBER)
State: closed
Created: 2020-09-22T09:47:23Z

When a dataset is formatted with a list of columns that `__getitem__` should return, then calling `map` to add new columns doesn't add the new columns to this list. 

It caused `KeyError` issues in #620 

I changed the logic to add those new columns to the list that `__getitem__` should return.

Comments:
Title: Squad Metric Description & Feature Mismatch
Author: tshrjn (NONE)
State: closed
Created: 2020-09-22T09:07:00Z

The [description](https://github.com/huggingface/datasets/blob/master/metrics/squad/squad.py#L39) doesn't mention `answer_start` in squad. However the `datasets.features` require [it](https://github.com/huggingface/datasets/blob/master/metrics/squad/squad.py#L68). It's also not used in the evaluation.

Comments:
Thanks for reporting !
There indeed a mismatch between the features and the kwargs description

I believe `answer_start` was added to match the squad dataset format for consistency, even though it is not used in the metric computation. I think I'd rather keep it this way, so that you can just give `references=squad["answers"]` to `.compute()`.
Maybe we can just fix the description then.

But then providing the `answer_start` becomes mandatory since the format of the features is checked against the one provided in the squad [file](https://github.com/huggingface/datasets/pull/658/files).
Title: Use multiprocess from pathos for multiprocessing
Author: lhoestq (MEMBER)
State: closed
Created: 2020-09-21T16:12:19Z

[Multiprocess](https://github.com/uqfoundation/multiprocess) (from the [pathos](https://github.com/uqfoundation/pathos) project) allows to use lambda functions in multiprocessed map.
It was suggested to use it by @kandorm.

We're already using dill which is its only dependency.

Comments:
We can just install multiprocess actually, I'll change that

Just an FYI: I remember that I wanted to try pathos a couple of years back and I ran into issues considering cross-platform; the code would just break on Windows. If I can verify this PR by running CPU tests on Windows, let me know!

That's good to know thanks
I guess we can just wait for #644 to be merged first. I'm working on fixing the tests for windows

Looks like all the CI jobs on windows passed !
I also tested locally on my windows and it works great :) 

I think this is ready to merge, let me know if you have any remarks @thomwolf @BramVanroy
Title: added Winogrande debiased subset
Author: TevenLeScao (CONTRIBUTOR)
State: closed
Created: 2020-09-21T14:51:08Z

The [Winogrande](https://arxiv.org/abs/1907.10641) paper mentions a `debiased` subset that wasn't in the first release; this PR adds it.

Comments:
To fix the CI you just have to copy the dummy data to the 1.1.0 folder, and maybe create the dummy ones for the `debiased` configuration

Fixed! Thanks @lhoestq
Title: handle data alteration when trying type
Author: lhoestq (MEMBER)
State: closed
Created: 2020-09-21T10:41:49Z

Fix #649 

The bug came from the type inference that didn't handle a weird case in Pyarrow.
Indeed this code runs without error but alters the data in arrow:
```python
import pyarrow as pa

type = pa.struct({"a": pa.struct({"b": pa.string()})})
array_with_altered_data = pa.array([{"a": {"b": "foo", "c": "bar"}}] * 10, type=type)
print(array_with_altered_data[0].as_py())
# {'a': {'b': 'foo'}} -> the sub-field "c" is missing
```
(I don't know if this is intended in pyarrow tbh)

We didn't take this case into account during type inference. By default it was keeping old features and maybe alter data.
To fix that I added a line that checks that the first element of the array is not altered.

Comments:
Title: Problem with JSON dataset format
Author: vikigenius (NONE)
State: open
Created: 2020-09-20T23:57:14Z

I have a local json dataset with the following form.

{
    'id01234': {'key1': value1, 'key2': value2, 'key3': value3},
    'id01235': {'key1': value1, 'key2': value2, 'key3': value3},
    .
    .
    .
    'id09999': {'key1': value1, 'key2': value2, 'key3': value3}
}
Note that instead of a list of records it's basically a dictionary of key value pairs with the keys being the record_ids and the values being the corresponding record.

Reading this with json:

```
data = datasets.load('json', data_files='path_to_local.json')
```
Throws an error and asks me to chose a field. What's the right way to handle this?

Comments:
Currently the `json` dataset doesn't support this format unfortunately.
However you could load it with
```python
from datasets import Dataset
import pandas as pd

df = pd.read_json("path_to_local.json", orient="index")
dataset = Dataset.from_pandas(df)
```

or you can make a custom dataset script as explained in doc here: https://huggingface.co/docs/datasets/add_dataset.html
Title: dummy data testing can't test datasets using `dl_manager.extract` in `_split_generators`
Author: richarddwang (CONTRIBUTOR)
State: closed
Created: 2020-09-19T11:07:03Z

Hi, I recently want to add a dataset whose source data is like this
```
openwebtext.tar.xz
  |__ openwebtext
         |__subset000.xz
         |     |__ ....txt
         |     |__ ....txt
         |     ...
         |__ subset001.xz
         |
         ....
```
So I wrote `openwebtext.py` like this
```
 def _split_generators(self, dl_manager):
        dl_dir = dl_manager.download_and_extract(_URL)
        owt_dir = os.path.join(dl_dir, 'openwebtext')
        subset_xzs = [
            os.path.join(owt_dir, file_name) for file_name in os.listdir(owt_dir) if file_name.endswith('xz') # filter out ...xz.lock
        ]
        ex_dirs = dl_manager.extract(subset_xzs, num_proc=round(os.cpu_count()*0.75))
        nested_txt_files = [ 
          [ 
            os.path.join(ex_dir,txt_file_name) for txt_file_name in os.listdir(ex_dir) if txt_file_name.endswith('txt')
          ] for ex_dir in ex_dirs
        ]
        txt_files = chain(*nested_txt_files)
        return [
            datasets.SplitGenerator(
                name=datasets.Split.TRAIN, gen_kwargs={"txt_files": txt_files}
            ),
        ]
```
All went good, I can load and use real openwebtext, except when I try to test with dummy data. The problem is  `MockDownloadManager.extract` do nothing, so `ex_dirs = dl_manager.extract(subset_xzs)` won't decompress `subset_xxx.xz`s for me.

How should I do ? Or you can modify `MockDownloadManager` to make it like a real `DownloadManager` ?

Comments:
Hi :) 
In your dummy data zip file you can just have `subset000.xz` as directories instead of compressed files.
Let me know if it helps

Thanks for your comment @lhoestq ,
Just for confirmation, changing dummy data like this won't make dummy test test the functionality to extract `subsetxxx.xz` but actually kind of circumvent it. But since we will test the real data so it is ok ?

Yes it's fine for now. We plan to add a job for slow tests.
And at one point we'll also do another pass on the dummy data handling and consider extracting files.

Thanks for the confirmation.
Also the suggestion works. Thank you.
Title: Inconsistent behavior in map
Author: krandiash (NONE)
State: closed
Created: 2020-09-19T08:41:12Z
Labels: bug

I'm observing inconsistent behavior when applying .map(). This happens specifically when I'm incrementally adding onto a feature that is a nested dictionary. Here's a simple example that reproduces the problem.

```python
import datasets

# Dataset with a single feature called 'field' consisting of two examples
dataset = datasets.Dataset.from_dict({'field': ['a', 'b']})
print(dataset[0])
# outputs
{'field': 'a'}

# Map this dataset to create another feature called 'otherfield', which is a dictionary containing a key called 'capital'
dataset = dataset.map(lambda example: {'otherfield': {'capital': example['field'].capitalize()}})
print(dataset[0])
# output is okay
{'field': 'a', 'otherfield': {'capital': 'A'}}

# Now I want to map again to modify 'otherfield', by adding another key called 'append_x' to the dictionary under 'otherfield'
print(dataset.map(lambda example: {'otherfield': {'append_x': example['field'] + 'x'}})[0])
# printing out the first example after applying the map shows that the new key 'append_x' doesn't get added
# it also messes up the value stored at 'capital'
{'field': 'a', 'otherfield': {'capital': None}}

# Instead, I try to do the same thing by using a different mapped fn
print(dataset.map(lambda example:  {'otherfield': {'append_x': example['field'] + 'x', 'capital': example['otherfield']['capital']}})[0])
# this preserves the value under capital, but still no 'append_x'
{'field': 'a', 'otherfield': {'capital': 'A'}}

# Instead, I try to pass 'otherfield' to remove_columns
print(dataset.map(lambda example:  {'otherfield': {'append_x': example['field'] + 'x', 'capital': example['otherfield']['capital']}}, remove_columns=['otherfield'])[0])
# this still doesn't fix the problem
{'field': 'a', 'otherfield': {'capital': 'A'}}

# Alternately, here's what happens if I just directly map both 'capital' and 'append_x' on a fresh dataset.

# Recreate the dataset
dataset = datasets.Dataset.from_dict({'field': ['a', 'b']})
# Now map the entire 'otherfield' dict directly, instead of incrementally as before
print(dataset.map(lambda example:  {'otherfield': {'append_x': example['field'] + 'x', 'capital': example['field'].capitalize()}})[0])
# This looks good!
{'field': 'a', 'otherfield': {'append_x': 'ax', 'capital': 'A'}}
```

This might be a new issue, because I didn't see this behavior in the `nlp` library. 

Any help is appreciated!

Comments:
Thanks for reporting !

This issue must have appeared when we refactored type inference in `nlp`
By default the library tries to keep the same feature types when applying `map` but apparently it has troubles with nested structures. I'll try to fix that next week
Title: Cannot download dataset_info.json
Author: chiyuzhang94 (NONE)
State: closed
Created: 2020-09-19T01:35:15Z

I am running my job on a cloud server where does not provide for connections from the standard compute nodes to outside resources. Hence, when I use `dataset.load_dataset()` to load data, I got an error like this:

```
ConnectionError: Couldn't reach https://storage.googleapis.com/huggingface-nlp/cache/datasets/text/default-53ee3045f07ba8ca/0.0.0/dataset_info.json
```

I tried to open this link manually, but I cannot access this file. How can I download this file and pass it through `dataset.load_dataset()` manually?

Versions:
Python version 3.7.3
PyTorch version 1.6.0
TensorFlow version 2.3.0
datasets version: 1.0.1

Comments:
Thanks for reporting !
We should add support for servers without internet connection indeed
I'll do that early next week

Thanks, @lhoestq !
Please let me know when it is available. 

Right now the recommended way is to create the dataset on a server with internet connection and then to save it and copy the serialized dataset to the server without internet connection.

#652 should allow you to load text/json/csv/pandas datasets without an internet connection **IF** you've the dataset script locally.

Example: 
If you have `datasets/text/text.py` locally, then you can do `load_dataset("./datasets/text", data_files=...)`
Title: Fix docs typos
Author: mariosasko (COLLABORATOR)
State: closed
Created: 2020-09-18T19:32:27Z

This PR fixes few typos in the docs and the error in the code snippet in the set_format section in docs/source/torch_tensorflow.rst. `torch.utils.data.Dataloader` expects padded batches so it throws an error due to not being able to stack the unpadded tensors. If we follow the Quick tour from the docs where they add the `truncation=True, padding='max_length'` arguments to the tokenizer before passing data to Dataloader, we can easily fix the issue.

Comments:
Title: Don't use take on dataset table in pyarrow 1.0.x
Author: lhoestq (MEMBER)
State: closed
Created: 2020-09-18T17:31:34Z

Fix #615

Comments:
I tried lower batch sizes and it didn't accelerate filter (quite the opposite actually).
The slow-down also appears for pyarrow 0.17.1 for some reason, not sure it comes from these changes

I just checked the benchmarks of other PRs and some of them had 300s (!!) for filter. This needs some investigation..

Merging this one since it's not the cause of the the slow down

@lhoestq What might be the reason for the slowdown? When I was training large batchsize, the slowdown was obvious, and my original assumption was to increase write_batch_size
Title: Better windows support
Author: lhoestq (MEMBER)
State: closed
Created: 2020-09-18T17:17:36Z

There are a few differences in the behavior of python and pyarrow on windows.

For example there are restrictions when accessing/deleting files that are open

Fix #590

Comments:
This PR is ready :)
It brings official support for windows.

Some tests `AWSDatasetTest` are failing.
This is because I had to fix a few datasets that were not compatible with windows.
These test will pass once they got merged on master :)
Title: Add Polyglot-NER Dataset
Author: joeddav (CONTRIBUTOR)
State: closed
Created: 2020-09-18T13:21:44Z

Adds the [Polyglot-NER dataset](https://sites.google.com/site/rmyeid/projects/polylgot-ner) with named entity tags for 40 languages. I include separate configs for each language as well as a `combined` config which lumps them all together.

Comments:
Hi @joeddav thanks for adding this! (I did a long webarchive.org session to actually find that dataset a while ago).

One question: should we manually correct the labeling scheme to (at least) IOB1?

That means "LOC" will be converted to "I-LOC". IOB1 is not explict. mentioned in the paper, but it is used in the documentation:

https://polyglot.readthedocs.io/en/latest/NamedEntityRecognition.html

@stefan-it I went back and forth on this. My biggest problem with it is that once you are in IOB, there is the expectation that the beginning of new entities are marked with a `B-` (at least in the case of two back-to-back entities):
```
Today    O
Alice    I-PER
Bob      B-PER
and      O
I        O 
ate      O
lasagna  O
```
If we just prepend `I-` to everything, `Bob` would be incorrectly tagged `I-PER`, meaning `Bob Alice` is a single entity. The current format is bad but is at least clear that it does not contain that information.

But I could go either way if someone has a strong opinion.

Indeed I'm not sure we can convert them to IOB because of this issue. I'm fine with keeping it like that

I'll do a release later today, hopefully we can include this dataset in the release :)

Let me know if you need help with the dummy data

@lhoestq cool thanks, I think I've got it right now – just zipped them wrong. I'm running tests locally now and then will push.

@lhoestq set to merge?

@joeddav I'm fine with keeping the original labeling scheme :)
Title: GLUE/QQP dataset: NonMatchingChecksumError
Author: richarddwang (CONTRIBUTOR)
State: closed
Created: 2020-09-18T07:09:10Z

Hi @lhoestq , I know you are busy and there are also other important issues. But if this is easy to be fixed, I am shamelessly wondering if you can give me some help , so I can evaluate my models and restart with my developing cycle asap. 😚

datasets version: editable install of master at 9/17

`datasets.load_dataset('glue','qqp', cache_dir='./datasets')`

```
Downloading and preparing dataset glue/qqp (download: 57.73 MiB, generated: 107.02 MiB, post-processed: Unknown size, total: 164.75 MiB) to ./datasets/glue/qqp/1.0.0/7c99657241149a24692c402a5c3f34d4c9f1df5ac2e4c3759fadea38f6cb29c4...
---------------------------------------------------------------------------
NonMatchingChecksumError                  Traceback (most recent call last)
 in 
----> 1 datasets.load_dataset('glue','qqp', cache_dir='./datasets')

~/datasets/src/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, save_infos, script_version, **config_kwargs)
    609         download_config=download_config,
    610         download_mode=download_mode,
--> 611         ignore_verifications=ignore_verifications,
    612     )
    613 

~/datasets/src/datasets/builder.py in download_and_prepare(self, download_config, download_mode, ignore_verifications, try_from_hf_gcs, dl_manager, **download_and_prepare_kwargs)
    467                     if not downloaded_from_gcs:
    468                         self._download_and_prepare(
--> 469                             dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
    470                         )
    471                     # Sync info

~/datasets/src/datasets/builder.py in _download_and_prepare(self, dl_manager, verify_infos, **prepare_split_kwargs)
    527         if verify_infos:
    528             verify_checksums(
--> 529                 self.info.download_checksums, dl_manager.get_recorded_sizes_checksums(), "dataset source files"
    530             )
    531 

~/datasets/src/datasets/utils/info_utils.py in verify_checksums(expected_checksums, recorded_checksums, verification_name)
     37     if len(bad_urls) > 0:
     38         error_msg = "Checksums didn't match" + for_verification_name + ":\n"
---> 39         raise NonMatchingChecksumError(error_msg + str(bad_urls))
     40     logger.info("All the checksums matched successfully" + for_verification_name)
     41 

NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://dl.fbaipublicfiles.com/glue/data/QQP-clean.zip']
```

Comments:
Hi ! Sure I'll take a look
Title: Add ConLL-2000 dataset
Author: vblagoje (CONTRIBUTOR)
State: closed
Created: 2020-09-16T11:14:11Z

Adds ConLL-2000 dataset used for text chunking. See https://www.clips.uantwerpen.be/conll2000/chunking/ for details and [motivation](https://github.com/huggingface/transformers/pull/7041#issuecomment-692710948) behind this PR

Comments:
Title: Fix text delimiter
Author: lhoestq (MEMBER)
State: closed
Created: 2020-09-15T08:08:42Z

I changed the delimiter in the `text` dataset script.
It should fix the `pyarrow.lib.ArrowInvalid: CSV parse error` from #622 

I changed the delimiter to an unused ascii character that is not present in text files : `\b`

Comments:
Which OS are you using ?@abhi1nandy2

> Which OS are you using ?

PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
VERSION_CODENAME=stretch
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Do you mind sharing the data you used (or part of it), so I can try to reproduce ?
Or at least some info about the text file you're using ? (size, n of lines, encoding)

Lot of data, difficult to share. There are 46 shards, each having about 256000 lines. using `file` command gives this - `ASCII text, with very long lines`.

Ok I see, no problem :) 
I'll see what I can do

Could you just test with one single dummy text file with a few lines to see if you're having the issue ?
Also which version of `datasets` do you have ?
Title: straddling object straddles two block boundaries
Author: bharaniabhishek123 (NONE)
State: closed
Created: 2020-09-15T00:30:46Z

I am trying to read json data (it's an array with lots of dictionaries) and getting block boundaries issue as below : 

I tried calling read_json with readOptions but no luck .

```
table = json.read_json(fn)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyarrow/_json.pyx", line 246, in pyarrow._json.read_json
  File "pyarrow/error.pxi", line 122, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: straddling object straddles two block boundaries (try to increase block size?)
```

Comments:
sorry it's an apache arrow issue.
Title: Update GLUE URLs (now hosted on FB)
Author: jeswan (CONTRIBUTOR)
State: closed
Created: 2020-09-14T19:05:39Z

NYU is switching dataset hosting from Google to FB. This PR closes https://github.com/huggingface/datasets/issues/608 and is necessary for https://github.com/jiant-dev/jiant/issues/161. This PR updates the data URLs based on changes made in https://github.com/nyu-mll/jiant/pull/1112.

Note: rebased on huggingface/datasets

Comments:
Title: Mistakes in MLQA features names
Author: M-Salti (CONTRIBUTOR)
State: closed
Created: 2020-09-11T20:46:23Z

I think the following features in MLQA shouldn't be named the way they are:
1. `questions` (should be `question`)
2. `ids` (should be `id`)
3. `start` (should be `answer_start`)

The reasons I'm suggesting these features be renamed are:
* To make them consistent with other QA datasets like SQuAD, XQuAD, TyDiQA etc. and hence make it easier to concatenate multiple QA datasets.
* The features names are not the same as the ones provided in the original MLQA datasets (it uses the names I suggested).

I know these columns can be renamed using  using `Dataset.rename_column_`, `questions` and `ids` can be easily renamed but `start` on the other hand is annoying to rename since it's nested inside the feature `answers`.

Comments:
Indeed you're right ! Thanks for reporting that

Could you open a PR to fix the features names ?
Title: Add CoNLL-2003 shared task dataset
Author: vblagoje (CONTRIBUTOR)
State: closed
Created: 2020-09-11T10:02:30Z

Please consider adding CoNLL-2003 shared task dataset as it's beneficial for token classification tasks. The motivation behind this PR is the [PR](https://github.com/huggingface/transformers/pull/7041) in the transformers project. This dataset would be not only useful for the usual run-of-the-mill NER tasks but also for syntactic chunking and part-of-speech (POS) tagging.

Comments:
I think we should somewhere mention, that is the dataset in IOB2 tagging scheme, whereas the original dataset uses IOB1 :)

Indeed this is something we want to mention.

If would want to add more details about the IOB1->2 change, feel free to ignore my suggestions and edit the description + update the dataset_info

@lhoestq do you want me to update it or you'll update it. I am ok either way

The best would be to mention this change in the description and then update the dataset_info.json file.
Could you do that if you don't mind ?

Then it should be ready to merge :)

Thanks again for adding this dataset !

No problem @lhoestq I'll do the update

@lhoestq please check if 847addf is exactly what we want

Is the German task also part of this? If not, can it be accessed via the Datasets library?
Title: Update GLUE URLs (now hosted on FB)
Author: jeswan (CONTRIBUTOR)
State: closed
Created: 2020-09-10T18:16:32Z

NYU is switching dataset hosting from Google to FB. This PR closes https://github.com/huggingface/datasets/issues/608 and is necessary for https://github.com/jiant-dev/jiant/issues/161. This PR updates the data URLs based on changes made in https://github.com/nyu-mll/jiant/pull/1112.

Comments:
Thanks for opening this PR :) 

We changed the name of the lib from nlp to datasets yesterday.
Could you rebase from master and re-generate the dataset_info.json file to fix the name changes ?

Rebased changes here: https://github.com/huggingface/datasets/pull/626
Title: Don't use the old NYU GLUE dataset URLs
Author: jeswan (CONTRIBUTOR)
State: closed
Created: 2020-09-10T17:47:02Z

NYU is switching dataset hosting from Google to FB. Initial changes to `datasets` are in https://github.com/jeswan/nlp/commit/b7d4a071d432592ded971e30ef73330529de25ce. What tests do you suggest I run before opening a PR?

See: https://github.com/jiant-dev/jiant/issues/161 and https://github.com/nyu-mll/jiant/pull/1112

Comments:
Feel free to open the PR ;)
Thanks for updating the dataset_info.json file !
Title: Set scripts version to master
Author: lhoestq (MEMBER)
State: closed
Created: 2020-09-10T10:47:44Z

By default the scripts version is master, so that if the library is installed with 
```
pip install git+http://github.com/huggingface/nlp.git
```
or
```
git clone http://github.com/huggingface/nlp.git
pip install -e ./nlp
```

will use the latest scripts, and not the ones from the previous version.

Comments:
Title: apply offset to indices in multiprocessed map
Author: lhoestq (MEMBER)
State: closed
Created: 2020-09-10T08:54:30Z

Fix #597 

I fixed the indices by applying an offset.
I added the case to our tests to make sure it doesn't happen again.

I also added the message proposed by @thomwolf in #597 

```python
>>> d.select(range(10)).map(fn, with_indices=True, batched=True, num_proc=2, load_from_cache_file=False)
Done writing 10 indices in 80 bytes .
Testing the mapped function outputs
[0, 1]
Testing finished, running the mapping function on the dataset
Done writing 5 indices in 41 bytes .
Done writing 5 indices in 41 bytes .
Spawning 2 processes
[0, 1, 2, 3, 4]
[5, 6, 7, 8, 9]
#0: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 377.90ba/s]
#1: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 378.92ba/s]
Concatenating 2 shards from multiprocessing

# Dataset(features: {'label': ClassLabel(num_classes=2, names=['neg', 'pos'], names_file=None, id=None), 'text': Value(dtype='string', id=None)}, num_rows: 10)
```

Comments:
Title: Add MATINF dataset
Author: JetRunner (CONTRIBUTOR)
State: closed
Created: 2020-09-10T03:31:09Z

@lhoestq The command to create metadata failed. I guess it's because the zip is not downloaded from a remote address? How to solve that? Also the CI fails and I don't know how to fix that :(

Comments:
Hi ! sorry for the late response

Could you try to rebase from master ? We changed the named of the library last week so you have to include this change in your code.

Can you give me more details about the error you get when running the cli command ?

Note that in case of a manual download you have to specify the directory where you downloaded the data with `--data_dir <path/to/the/directory>`

I fucked up the Git rebase lol. Closing it.
Title: Indices incorrect with multiprocessing
Author: joeddav (CONTRIBUTOR)
State: closed
Created: 2020-09-09T19:50:56Z

When `num_proc` > 1, the indices argument passed to the map function is incorrect:

```python
d = load_dataset('imdb', split='test[:1%]')

def fn(x, inds):
    print(inds)
    return x

d.select(range(10)).map(fn, with_indices=True, batched=True)
# [0, 1]
# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

d.select(range(10)).map(fn, with_indices=True, batched=True, num_proc=2)
# [0, 1]
# [0, 1]
# [0, 1, 2, 3, 4]
# [0, 1, 2, 3, 4]
```

As you can see, the subset passed to each thread is indexed from 0 to N which doesn't reflect their positions in `d`.

Comments:
I fixed a bug that could cause this issue earlier today. Could you pull the latest version and try again ?

Still the case on master.
I guess we should have an offset in the multi-procs indeed (hopefully it's enough).

Also, side note is that we should add some logging before the "test" to say we are testing the function otherwise its confusing for the user to see two outputs I think. Proposal (see the "Testing the mapped function outputs:" lines):
```
>>> d.select(range(10)).map(fn, with_indices=True, batched=True, num_proc=2)
Done writing 10 indices in 80 bytes .
Done writing 5 indices in 41 bytes .
Done writing 5 indices in 41 bytes .
Spawning 2 processes
Testing the mapped function outputs:
inds: [0, 1]
inds: [0, 1]
Testing finished, running the mapped function on the dataset:
#0:   0%|                                                                                                                                                                    | 0/1 [00:00<?, ?ba/s]
inds: [0, 1, 2, 3, 4]                                                                                                                                                                             inds: [0, 1, 2, 3, 4]                                                                                                                                                         | 0/1 [00:00<?, ?ba/s]
#0: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1321.04ba/s]
#1: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1841.22ba/s]
Concatenating 2 shards from multiprocessing
Dataset(features: {'text': Value(dtype='string', id=None), 'label': ClassLabel(num_classes=2, names=['neg', 'pos'], names_file=None, id=None)}, num_rows: 10)
```
Title: [style/quality] Moving to isort 5.0.0 + style/quality on datasets and metrics
Author: thomwolf (MEMBER)
State: closed
Created: 2020-09-09T15:47:21Z

Move the repo to isort 5.0.0.

Also start testing style/quality on datasets and metrics.

Specific rule: we allow F401 (unused imports) in metrics to be able to add imports to detect early on missing dependencies.
Maybe we could add this in datasets but while cleaning this I've seen many example of really unused imports in dataset so maybe it's better to have it as a line-by-line nova instead of a general rule like in metrics.

Comments:
Ready for review @lhoestq, just updated a few 156 files here
Title: `Dataset`/`DatasetDict` has no attribute 'save_to_disk'
Author: sudarshan85 (NONE)
State: closed
Created: 2020-09-09T15:01:52Z

Hi,

As the title indicates, both `Dataset` and `DatasetDict` classes don't seem to have the `save_to_disk` method.  While the file [`arrow_dataset.py`](https://github.com/huggingface/nlp/blob/34bf0b03bfe03e7f77b8fec1cd48f5452c4fc7c1/src/nlp/arrow_dataset.py) in the repo here has the method, the file `arrow_dataset.py` which is saved after `pip install nlp -U` in my `conda` environment DOES NOT contain the `save_to_disk` method. I even tried `pip install git+https://github.com/huggingface/nlp.git ` and still no luck. Do I need to install the library in another way?

Comments:
`pip install git+https://github.com/huggingface/nlp.git` should have done the job.

Did you uninstall `nlp` before installing from github ?

> Did you uninstall `nlp` before installing from github ?

I did not. I created a new environment and installed `nlp` directly from `github` and it worked!

Thanks.
Title: GermEval 2014: new download urls
Author: stefan-it (CONTRIBUTOR)
State: closed
Created: 2020-09-09T10:07:29Z

Hi,

unfortunately, the download links for the GermEval 2014 dataset have changed: they're now located on a Google Drive.

I changed the URLs and bump version from 1.0.0 to 2.0.0.

Comments:
/cc: @vblagoje

Closing this one as #594 is merged (same changes except the dummy data hack)

Awesome @stefan-it ! @lhoestq how soon can I use the fixed GermEval dataset in HF token classification examples?

I've manually updated the script on S3, so you can actually use it right now with
```python
from nlp import load_dataset

germeval = load_dataset("germeval_14")
```

not sure if it's used in token classification examples already

Awesome. Not used yet but I am going to use it now. I've been working on an update for token classification examples and this was a missing piece. Thanks @stefan-it @lhoestq
Title: The process cannot access the file because it is being used by another process (windows)
Author: saareliad (NONE)
State: closed
Created: 2020-09-09T07:01:36Z

Hi, I consistently get the following error when developing in my PC (windows 10):

```
    train_dataset = train_dataset.map(convert_to_features, batched=True)
  File "C:\Users\saareliad\AppData\Local\Continuum\miniconda3\envs\py38\lib\site-packages\nlp\arrow_dataset.py", line 970, in map
    shutil.move(tmp_file.name, cache_file_name)
  File "C:\Users\saareliad\AppData\Local\Continuum\miniconda3\envs\py38\lib\shutil.py", line 803, in move
    os.unlink(src)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\saareliad\\.cache\\huggingface\\datasets\\squad\\plain_text\\1.0.0\\408a8fa46a1e2805445b793f1022e743428ca739a34809fce872f0c7f17b44ab\\tmpsau1bep1'

```

Comments:
Hi, which version of `nlp` are you using?

By the way we'll be releasing today a significant update fixing many issues (but also comprising a few breaking changes).
You can see more informations here #545 and try it by installing from source from the master branch.

I'm using version 0.4.0.



Ok, it's probably fixed on master. Otherwise if you can give me a fully self-contained exemple to reproduce the error, I can try to investigate.

I get the same behavior, on Windows, when `map`ping a function to a loaded dataset. 
The error doesn't occur if I re-run the cell a second time though! 
I'm on version 1.0.1.

This is going to be fixed by #644 

@saareliad I got the same issue that troubled me quite a while. Unfortunately, there are no good answers to this issue online, I tried it on Linux and that's absolutely fine. After hacking the source code, I solved this problem as follows.

In the source code file: arrow_dataset.py -> _map_single(...)

change
```python
if update_data and tmp_file is not None:
    shutil.move(tmp_file.name, cache_file_name)
```
to
```python
tmp_file.close()
if update_data and tmp_file is not None:
    shutil.move(tmp_file.name, cache_file_name)
```

Then it works without needing multiple times runs to avoid the permission error.
I know this solution is unusual since it changes the source code. Hopefully, the lib's contributors can have better solutions in the future.


@wangcongcong123 thanks  for sharing.
(BTW I also solved it locally on windows by putting the problematic line under try except and not using cache... On windows I just needed 1% of the dataset anyway)
Title: Cannot use nlp.load_dataset text, AttributeError: module 'nlp.utils' has no attribute 'logging'
Author: ksjae (NONE)
State: closed
Created: 2020-09-09T06:46:53Z

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/anaconda3/envs/pytorch/lib/python3.7/site-packages/nlp/load.py", line 533, in load_dataset
    builder_cls = import_main_class(module_path, dataset=True)
  File "/root/anaconda3/envs/pytorch/lib/python3.7/site-packages/nlp/load.py", line 61, in import_main_class
    module = importlib.import_module(module_path)
  File "/root/anaconda3/envs/pytorch/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/root/anaconda3/envs/pytorch/lib/python3.7/site-packages/nlp/datasets/text/5dc629379536c4037d9c2063e1caa829a1676cf795f8e030cd90a537eba20c08/text.py", line 9, in <module>
    logger = nlp.utils.logging.get_logger(__name__)
AttributeError: module 'nlp.utils' has no attribute 'logging'
```

Occurs on the following code, or any code including the load_dataset('text'):
```
dataset = load_dataset("text", data_files=file_path, split="train")
dataset = dataset.map(lambda ex: tokenizer(ex["text"], add_special_tokens=True,
                                        truncation=True, max_length=args.block_size), batched=True)
dataset.set_format(type='torch', columns=['input_ids'])
return dataset
```

Comments:
Title: Use github versioning
Author: lhoestq (MEMBER)
State: closed
Created: 2020-09-07T14:58:15Z

Right now dataset scripts and metrics are downloaded from S3 which is in sync with master. It means that it's not currently possible to pin the dataset/metric script version.

To fix that I changed the download url from S3 to github, and adding a `version` parameter in `load_dataset` and `load_metric` to pin a certain version of the lib, as in #562

Comments:
I noticed that datasets like `cnn_dailymail` need the `version` parameter to be passed to its `config_kwargs`.
Shall we rename the `version` paramater in `load_dataset` ? Maybe `repo_version` or `script_version` ?
Title: ArrowIndexError on Dataset.select
Author: lhoestq (MEMBER)
State: closed
Created: 2020-09-07T14:36:29Z

If the indices table consists in several chunks, then `dataset.select` results in an `ArrowIndexError` error for pyarrow < 1.0.0

Example:

```python
from nlp import load_dataset

mnli = load_dataset("glue", "mnli", split="train")
shuffled = mnli.shuffle(seed=42)
mnli.select(list(range(len(mnli))))
```

raises:
```python
---------------------------------------------------------------------------
ArrowIndexError                           Traceback (most recent call last)
<ipython-input-64-006a5d38d418> in <module>
----> 1 mnli.shuffle(seed=42).select(list(range(len(mnli))))

~/Desktop/hf/nlp/src/nlp/fingerprint.py in wrapper(*args, **kwargs)
    161             # Call actual function
    162 
--> 163             out = func(self, *args, **kwargs)
    164 
    165             # Update fingerprint of in-place transforms + update in-place history of transforms

~/Desktop/hf/nlp/src/nlp/arrow_dataset.py in select(self, indices, keep_in_memory, indices_cache_file_name, writer_batch_size, new_fingerprint)
   1653         if self._indices is not None:
   1654             if PYARROW_V0:
-> 1655                 indices_array = self._indices.column(0).chunk(0).take(indices_array)
   1656             else:
   1657                 indices_array = self._indices.column(0).take(indices_array)

~/.virtualenvs/hf-datasets/lib/python3.7/site-packages/pyarrow/array.pxi in pyarrow.lib.Array.take()

~/.virtualenvs/hf-datasets/lib/python3.7/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status()

ArrowIndexError: take index out of bounds
```

This is because the `take` method is only done on the first chunk which only contains 1000 elements by default (mnli has ~400 000 elements).

Shall we change that to use 
```python
pa.concat_tables(self._indices._indices.slice(i, 1) for i in indices_array)
```
instead of `take` ? @thomwolf

Comments:
Title: Allow for PathLike objects
Author: BramVanroy (CONTRIBUTOR)
State: closed
Created: 2020-09-07T13:54:51Z

Using PathLike objects as input for `load_dataset` does not seem to work. The following will throw an error.

```python
files = list(Path(r"D:\corpora\yourcorpus").glob("*.txt"))
dataset = load_dataset("text", data_files=files)
```

Traceback:

```
Traceback (most recent call last):
  File "C:/dev/python/dutch-simplification/main.py", line 7, in <module>
    dataset = load_dataset("text", data_files=files)
  File "C:\Users\bramv\.virtualenvs\dutch-simplification-nbNdqK9u\lib\site-packages\nlp\load.py", line 548, in load_dataset
    builder_instance.download_and_prepare(
  File "C:\Users\bramv\.virtualenvs\dutch-simplification-nbNdqK9u\lib\site-packages\nlp\builder.py", line 470, in download_and_prepare
    self._save_info()
  File "C:\Users\bramv\.virtualenvs\dutch-simplification-nbNdqK9u\lib\site-packages\nlp\builder.py", line 564, in _save_info
    self.info.write_to_directory(self._cache_dir)
  File "C:\Users\bramv\.virtualenvs\dutch-simplification-nbNdqK9u\lib\site-packages\nlp\info.py", line 149, in write_to_directory
    self._dump_info(f)
  File "C:\Users\bramv\.virtualenvs\dutch-simplification-nbNdqK9u\lib\site-packages\nlp\info.py", line 156, in _dump_info
    file.write(json.dumps(asdict(self)).encode("utf-8"))
  File "c:\users\bramv\appdata\local\programs\python\python38\lib\json\__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "c:\users\bramv\appdata\local\programs\python\python38\lib\json\encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "c:\users\bramv\appdata\local\programs\python\python38\lib\json\encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
TypeError: keys must be str, int, float, bool or None, not WindowsPath
```

We have to cast to a string explicitly to make this work. It would be nicer if we could actually use PathLike objects.

```python
files = [str(f) for f in Path(r"D:\corpora\wablieft").glob("*.txt")]
```

Comments:
Title: nlp re-creates already-there caches when using a script, but not within a shell
Author: TevenLeScao (CONTRIBUTOR)
State: closed
Created: 2020-09-07T10:23:50Z

`nlp` keeps creating new caches for the same file when launching `filter` from a script, and behaves correctly from within the shell.

Example: try running

```
import nlp

hans_easy_data = nlp.load_dataset('hans', split="validation").filter(lambda x: x['label'] == 0)
hans_hard_data = nlp.load_dataset('hans', split="validation").filter(lambda x: x['label'] == 1)
```

twice. If launched from a `file.py` script, the cache will be re-created the second time. If launched as 3 shell/`ipython` commands, `nlp` will correctly re-use the cache.
As observed with @lhoestq.

Comments:
Couln't reproduce on my side :/ 
let me know if you manage to reproduce on another env (colab for example)

Fixed with a clean re-install!