multiprocessing in dataset map "can only test a child process"

It looks like an issue with wandb/tqdm here.
We're using the `multiprocess` library instead of the `multiprocessing` builtin python package to support various types of mapping functions. Maybe there's some sort of incompatibility.

Could you make a minimal script to reproduce or a google colab ?

hi facing the same issue here -

`AssertionError: Caught AssertionError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/usr/lib/python3.6/logging/__init__.py", line 996, in emit
    stream.write(msg)
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/lib/redirect.py", line 100, in new_write
    cb(name, data)
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/wandb_run.py", line 723, in _console_callback
    self._backend.interface.publish_output(name, data)
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/interface/interface.py", line 153, in publish_output
    self._publish_output(o)
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/interface/interface.py", line 158, in _publish_output
    self._publish(rec)
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/interface/interface.py", line 456, in _publish
    if self._process and not self._process.is_alive():
  File "/usr/lib/python3.6/multiprocessing/process.py", line 134, in is_alive
    assert self._parent_pid == os.getpid(), 'can only test a child process'
AssertionError: can only test a child process

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop
    data = fetcher.fetch(index)
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "<ipython-input-8-a4d9a08d114e>", line 20, in __getitem__
    return_token_type_ids=True
  File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils_base.py", line 2405, in encode_plus
    **kwargs,
  File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils_base.py", line 2125, in _get_padding_truncation_strategies
    "Truncation was not explicitly activated but `max_length` is provided a specific value, "
  File "/usr/lib/python3.6/logging/__init__.py", line 1320, in warning
    self._log(WARNING, msg, args, **kwargs)
  File "/usr/lib/python3.6/logging/__init__.py", line 1444, in _log
    self.handle(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 1454, in handle
    self.callHandlers(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 1516, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 865, in handle
    self.emit(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 1000, in emit
    self.handleError(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 917, in handleError
    sys.stderr.write('--- Logging error ---\n')
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/lib/redirect.py", line 100, in new_write
    cb(name, data)
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/wandb_run.py", line 723, in _console_callback
    self._backend.interface.publish_output(name, data)
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/interface/interface.py", line 153, in publish_output
    self._publish_output(o)
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/interface/interface.py", line 158, in _publish_output
    self._publish(rec)
  File "/usr/local/lib/python3.6/dist-packages/wandb/sdk/interface/interface.py", line 456, in _publish
    if self._process and not self._process.is_alive():
  File "/usr/lib/python3.6/multiprocessing/process.py", line 134, in is_alive
    assert self._parent_pid == os.getpid(), 'can only test a child process'
AssertionError: can only test a child process`

It looks like this warning :
"Truncation was not explicitly activated but max_length is provided a specific value, "
is not handled well by wandb.

The error occurs when calling the tokenizer.
Maybe you can try to specify `truncation=True` when calling the tokenizer to remove the warning ?
Otherwise I don't know why wandb would fail on a warning. Maybe one of its logging handlers have some issues with the logging of tokenizers. Maybe @n1t0 knows more about this ?

I'm having a similar issue but when I try to do multiprocessing with the `DataLoader`

Code to reproduce:

```
from datasets import load_dataset

book_corpus = load_dataset('bookcorpus', 'plain_text', cache_dir='/home/ad/Desktop/bookcorpus', split='train[:1%]')
book_corpus = book_corpus.map(encode, batched=True, num_proc=20, load_from_cache_file=True, batch_size=5000)
book_corpus.set_format(type='torch', columns=['text', "input_ids", "attention_mask", "token_type_ids"])

from transformers import DataCollatorForWholeWordMask
from transformers import Trainer, TrainingArguments

data_collator = DataCollatorForWholeWordMask(
    tokenizer=tokenizer, mlm=True, mlm_probability=0.15)

training_args = TrainingArguments(
    output_dir="./mobile_linear_att_8L_128_128_03layerdrop_shared",
    overwrite_output_dir=True,
    num_train_epochs=1,
    per_device_train_batch_size=64,
    save_steps=50,
    save_total_limit=2,
    logging_first_step=True,
    warmup_steps=100,
    logging_steps=50,
    gradient_accumulation_steps=1,
    fp16=True,
    **dataloader_num_workers=10**,
)

trainer = Trainer(
    model=model,
    args=training_args,
    data_collator=data_collator,
    train_dataset=book_corpus,
    tokenizer=tokenizer)

trainer.train()
```

```
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<timed eval> in <module>

~/anaconda3/envs/tfm/lib/python3.6/site-packages/transformers/trainer.py in train(self, model_path, trial)
    869             self.control = self.callback_handler.on_epoch_begin(self.args, self.state, self.control)
    870
--> 871             for step, inputs in enumerate(epoch_iterator):
    872
    873                 # Skip past any already trained steps if resuming training

~/anaconda3/envs/tfm/lib/python3.6/site-packages/torch/utils/data/dataloader.py in __next__(self)
    433         if self._sampler_iter is None:
    434             self._reset()
--> 435         data = self._next_data()
    436         self._num_yielded += 1
    437         if self._dataset_kind == _DatasetKind.Iterable and \

~/anaconda3/envs/tfm/lib/python3.6/site-packages/torch/utils/data/dataloader.py in _next_data(self)
   1083             else:
   1084                 del self._task_info[idx]
-> 1085                 return self._process_data(data)
   1086
   1087     def _try_put_index(self):

~/anaconda3/envs/tfm/lib/python3.6/site-packages/torch/utils/data/dataloader.py in _process_data(self, data)
   1109         self._try_put_index()
   1110         if isinstance(data, ExceptionWrapper):
-> 1111             data.reraise()
   1112         return data
   1113

~/anaconda3/envs/tfm/lib/python3.6/site-packages/torch/_utils.py in reraise(self)
    426             # have message field
    427             raise self.exc_type(message=msg)
--> 428         raise self.exc_type(msg)
    429
    430

AssertionError: Caught AssertionError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/datasets/arrow_dataset.py", line 1087, in __getitem__
    format_kwargs=self._format_kwargs,
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/datasets/arrow_dataset.py", line 1074, in _getitem
    format_kwargs=format_kwargs,
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/datasets/arrow_dataset.py", line 890, in _convert_outputs
    v = map_nested(command, v, **map_nested_kwargs)
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/datasets/utils/py_utils.py", line 225, in map_nested
    return function(data_struct)
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/datasets/arrow_dataset.py", line 851, in command
    return torch.tensor(x, **format_kwargs)
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/warnings.py", line 101, in _showwarnmsg
    _showwarnmsg_impl(msg)
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/warnings.py", line 30, in _showwarnmsg_impl
    file.write(text)
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/wandb/sdk/lib/redirect.py", line 100, in new_write
    cb(name, data)
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/wandb/sdk/wandb_run.py", line 723, in _console_callback
    self._backend.interface.publish_output(name, data)
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/wandb/sdk/interface/interface.py", line 153, in publish_output
    self._publish_output(o)
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/wandb/sdk/interface/interface.py", line 158, in _publish_output
    self._publish(rec)
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/wandb/sdk/interface/interface.py", line 456, in _publish
    if self._process and not self._process.is_alive():
  File "/home/ad/anaconda3/envs/tfm/lib/python3.6/multiprocessing/process.py", line 134, in is_alive
    assert self._parent_pid == os.getpid(), 'can only test a child process'
AssertionError: can only test a child process
```

As a workaround I have commented line 456 and 457 in `/home/ad/anaconda3/envs/tfm/lib/python3.6/site-packages/wandb/sdk/interface/interface.py`

Isn't it more the pytorch warning on the use of non-writable memory for tensor that trigger this here @lhoestq? (since it seems to be a warning triggered in `torch.tensor()`

Yep this time this is a warning from pytorch that causes wandb to not work properly.
Could this by a wandb issue ?

Hi @timothyjlaurent @gaceladri
If you're running `transformers` from `master` you can try setting the env var `WAND_DISABLE=true` (from https://github.com/huggingface/transformers/pull/9896) and try again ?
This issue might be related to https://github.com/huggingface/transformers/issues/9623

I have commented the lines that cause my code break. I'm now seeing my reports on Wandb and my code does not break. I am training now, so I will check probably in 6 hours. I suppose that setting wandb disable will work as well.

Using a dataset with a single 'text' field and a fast tokenizer in a jupyter notebook.

```
def tokenizer_fn(example):
    return tokenizer.batch_encode_plus(example['text'])

ds_tokenized = text_dataset.map(tokenizer_fn, batched=True, num_proc=6, remove_columns=['text'])
```

```
---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/multiprocess/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/datasets/arrow_dataset.py", line 156, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/datasets/fingerprint.py", line 163, in wrapper
    out = func(self, *args, **kwargs)
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/datasets/arrow_dataset.py", line 1510, in _map_single
    for i in pbar:
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/tqdm/notebook.py", line 228, in __iter__
    for obj in super(tqdm_notebook, self).__iter__(*args, **kwargs):
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/tqdm/std.py", line 1186, in __iter__
    self.close()
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/tqdm/notebook.py", line 251, in close
    super(tqdm_notebook, self).close(*args, **kwargs)
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/tqdm/std.py", line 1291, in close
    fp_write('')
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/tqdm/std.py", line 1288, in fp_write
    self.fp.write(_unicode(s))
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/wandb/sdk/lib/redirect.py", line 91, in new_write
    cb(name, data)
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/wandb/sdk/wandb_run.py", line 598, in _console_callback
    self._backend.interface.publish_output(name, data)
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/wandb/sdk/interface/interface.py", line 146, in publish_output
    self._publish_output(o)
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/wandb/sdk/interface/interface.py", line 151, in _publish_output
    self._publish(rec)
  File "/home/jovyan/share/users/tlaurent/invitae-bert/ve/lib/python3.6/site-packages/wandb/sdk/interface/interface.py", line 431, in _publish
    if self._process and not self._process.is_alive():
  File "/usr/lib/python3.6/multiprocessing/process.py", line 134, in is_alive
    assert self._parent_pid == os.getpid(), 'can only test a child process'
AssertionError: can only test a child process
"""
```


Add HoVer multi-hop fact verification dataset

dataset request

Requesting to add a new dataset

Hi @yjernite  I'm new but wanted to contribute. Has anyone already taken this problem and do you think it is suitable for newbies?

Hi @tenjjin! This dataset is still up for grabs! Here's the link with the guide to add it. You should play around with the library first (download and look at a few datasets), then follow the steps here:

https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md

## Adding a Dataset
- **Name:** HoVer
- **Description:** https://twitter.com/YichenJiang9/status/1326954363806429186 contains 20K claim verification examples
- **Paper:** https://arxiv.org/abs/2011.03088
- **Data:** https://hover-nlp.github.io/
- **Motivation:** There are still few multi-hop information extraction benchmarks (HotpotQA, which dataset wase based off, notwithstanding)

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


amazon description fields as bullets

One more minor formatting change to amazon reviews's description (in addition to #844). Just reformatting the fields to display as a bulleted list in markdown.


add newlines to amazon desc

Just a quick formatting fix to hopefully make it render nicer on Viewer


use_custom_baseline still produces errors for bertscore

metric bug

A bug in a metric script

Thanks for reporting ! That's a bug indeed
If you want to contribute, feel free to fix this issue and open a PR :)

This error is because of a mismatch between `datasets` and `bert_score`. With `datasets=1.1.2` and `bert_score>=0.3.6` it works ok. So `pip install -U bert_score` should fix the problem.

Thanks for the heads up @pvl and for the PR as well :)

Hello everyone,

I think the  problem is not solved:

```
from datasets import load_metric
metric=load_metric('bertscore')
metric.compute(
    predictions=predictions,
    references=references,
    lang='fr',
    rescale_with_baseline=True
)
TypeError: get_hash() missing 2 required positional arguments: 'use_custom_baseline' and 'use_fast_tokenizer'
```
This code is produced using `Python 3.6.9 datasets==1.1.2 and bert_score==0.3.10`

Hi ! This has been fixed by https://github.com/huggingface/datasets/pull/2770, we'll do a new release soon to make the fix available :)

In the meantime please use an older version of `bert_score`

`metric = load_metric('bertscore')`
`a1 = "random sentences"`
`b1 = "random sentences"`
`metric.compute(predictions = [a1], references = [b1], lang = 'en')`

`Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/stephen_chan/.local/lib/python3.6/site-packages/datasets/metric.py", line 393, in compute
    output = self._compute(predictions=predictions, references=references, **kwargs)
  File "/home/stephen_chan/.cache/huggingface/modules/datasets_modules/metrics/bertscore/361e597a01a41d6cf95d94bbfb01dea16261687abc0c6c74cc9930f80488f363/bertscore.py", line 108, in _compute
    hashcode = bert_score.utils.get_hash(model_type, num_layers, idf, rescale_with_baseline)
TypeError: get_hash() missing 1 required positional argument: 'use_custom_baseline'`

Adding 'use_custom_baseline = False' as an argument produces this error

`Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/stephen_chan/.local/lib/python3.6/site-packages/datasets/metric.py", line 393, in compute
    output = self._compute(predictions=predictions, references=references, **kwargs)
TypeError: _compute() got an unexpected keyword argument 'use_custom_baseline'`

This is on Ubuntu 18.04, Python 3.6.9, datasets version 1.1.2


How to enable `.map()` pre-processing pipelines to support multi-node parallelism?

Right now multiprocessing only runs on single node.

However it's probably possible to extend it to support multi nodes. Indeed we're using the `multiprocess` library from the `pathos` project to do multiprocessing in `datasets`, and `pathos` is made to support parallelism on several nodes. More info about pathos [on the pathos repo](https://github.com/uqfoundation/pathos).

If you're familiar with pathos or if you want to give it a try, it could be a nice addition to the library :)

Hi,

Currently, multiprocessing can be enabled for the `.map()` stages on a single node. However, in the case of multi-node training, (since more than one node would be available) I'm wondering if it's possible to extend the parallel processing among nodes, instead of only 1 node running the `.map()` while the other node is waiting for it to finish?

Thanks!


Can not reuse datasets already downloaded

It seems the process needs '/datasets.huggingface.co/datasets/datasets/wikipedia/wikipedia.py'
Where and how to assign this ```wikipedia.py``` after I manually download it ?

download the ```wikipedia.py``` at the working directory and go with ```dataset = load_dataset('wikipedia.py', '20200501.en')``` works.

Hello,
I need to connect to a frontal node (with http proxy, no gpu) before connecting to a gpu node (but no http proxy, so can not use wget so on).
I successfully downloaded and reuse the wikipedia datasets in a frontal node.
When I connect to the gpu node, I supposed to use the downloaded datasets from cache, but failed and end with time out error.

On frontal node:
```
>>> from datasets import load_dataset
>>> dataset = load_dataset('wikipedia', '20200501.en')
Reusing dataset wikipedia (/linkhome/rech/genini01/uua34ms/.cache/huggingface/datasets/wikipedia/20200501.en/1.0.0/f92599dfccab29832c442b82870fa8f6983e5b4ebbf5e6e2dcbe894e325339cd)
/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/torch/cuda/__init__.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  /pytorch/c10/cuda/CUDAFunctions.cpp:100.)
  return torch._C._cuda_getDeviceCount() > 0
```

On gpu node:
```
>>> from datasets import load_dataset
>>> dataset = load_dataset('wikipedia', '20200501.en')
Traceback (most recent call last):
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/urllib3/connection.py", line 309, in connect
    conn = self._new_conn()
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x14b7b73e4908>: Failed to establish a new connection: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/urllib3/util/retry.py", line 446, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /datasets.huggingface.co/datasets/datasets/wikipedia/wikipedia.py (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x14b7b73e4908>: Failed to establish a new connection: [Errno 110] Connection timed out',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/datasets/load.py", line 590, in load_dataset
    path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/datasets/load.py", line 264, in prepare_module
    head_hf_s3(path, filename=name, dataset=dataset)
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/datasets/utils/file_utils.py", line 200, in head_hf_s3
    return requests.head(hf_bucket_url(identifier=identifier, filename=filename, use_cdn=use_cdn, dataset=dataset))
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/requests/api.py", line 104, in head
    return request('head', url, **kwargs)
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/linkhome/rech/genini01/uua34ms/work/anaconda3/envs/pytorch_pip170_cuda102/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /datasets.huggingface.co/datasets/datasets/wikipedia/wikipedia.py (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x14b7b73e4908>: Failed to establish a new connection: [Errno 110] Connection timed out',))

```

Any advice?Thanks!


Update squad_v2.py

With this change all the checks are passed.

Change lines 100 and 102 to prevent overwriting ```predictions``` variable.


XSum dataset missing spaces between sentences

I noticed that the XSum dataset has no space between sentences. This could lead to worse results for anyone training or testing on it. Here's an example (0th entry in the test set):

`The London trio are up for best UK act and best album, as well as getting two nominations in the best song category."We got told like this morning 'Oh I think you're nominated'", said Dappy."And I was like 'Oh yeah, which one?' And now we've got nominated for four awards. I mean, wow!"Bandmate Fazer added: "We thought it's best of us to come down and mingle with everyone and say hello to the cameras. And now we find we've got four nominations."The band have two shots at the best song prize, getting the nod for their Tynchy Stryder collaboration Number One, and single Strong Again.Their album Uncle B will also go up against records by the likes of Beyonce and Kanye West.N-Dubz picked up the best newcomer Mobo in 2007, but female member Tulisa said they wouldn't be too disappointed if they didn't win this time around."At the end of the day we're grateful to be where we are in our careers."If it don't happen then it don't happen - live to fight another day and keep on making albums and hits for the fans."Dappy also revealed they could be performing live several times on the night.The group will be doing Number One and also a possible rendition of the War Child single, I Got Soul.The charity song is a  re-working of The Killers' All These Things That I've Done and is set to feature artists like Chipmunk, Ironik and Pixie Lott.This year's Mobos will be held outside of London for the first time, in Glasgow on 30 September.N-Dubz said they were looking forward to performing for their Scottish fans and boasted about their recent shows north of the border."We just done Edinburgh the other day," said Dappy."We smashed up an N-Dubz show over there. We done Aberdeen about three or four months ago - we smashed up that show over there! Everywhere we go we smash it up!"`


CNN/Dailymail Dataset Card

Link to the card page: https://github.com/mcmillanmajora/datasets/tree/cnn_dailymail_card/datasets/cnn_dailymail

One of the questions this dataset brings up is how we want to handle versioning of the cards to mirror versions of the dataset. The different versions of this dataset are used for different tasks (which may not be reflected in the versions that we currently have in the repo?), but it's only the structure that's changing rather than the content in this particular case, at least between versions 2.0.0 and 3.0.0.


AlloCiné dataset card

Link to the card page: https://github.com/mcmillanmajora/datasets/blob/allocine_card/datasets/allocine/README.md

There wasn't as much information available for this dataset, so I'm wondering what's the best way to address open questions about the dataset. For example, where did the list of films that the dataset creator used come from?

I'm also wondering how best to go about talking about limitations when so little is known about the data.


load_dataset with 'csv' is not working. while the same file is loading with 'text' mode or with pandas

dataset bug

A bug in a dataset script provided in the library

Which version of pyarrow do you have ? Could you try to update pyarrow and try again ?

Thanks for the fast response. I have the latest version '2.0.0' (I tried to update)
I am working with Python 3.8.5

I think that the issue is similar to this one:https://issues.apache.org/jira/browse/ARROW-9612
The problem is in arrow when the column data contains long strings.
Any ideas on how to bypass this?

We should expose the [`block_size` argument](https://arrow.apache.org/docs/python/generated/pyarrow.csv.ReadOptions.html#pyarrow.csv.ReadOptions) of Apache Arrow csv `ReadOptions` in the [script](https://github.com/huggingface/datasets/blob/master/datasets/csv/csv.py).

In the meantime you can specify yourself the `ReadOptions` config like this:
```python
import pyarrow.csv as pac   # PyArrow is installed with `datasets`

read_options = pac.ReadOptions(block_size=1e9)  # try to find the right value for your use-case
dataset = load_dataset('csv', data_files=files, read_options=read_options)
```

This did help to load the data. But the problem now is that I get:
ArrowInvalid: CSV parse error: Expected 5 columns, got 187

It seems that this change the parsing so I changed the table to tab-separated and tried to load it directly from pyarrow
But I got a similar error, again it loaded fine in pandas so I am not sure what to do.

Got almost the same error loading a ~5GB TSV file, first got the same error as OP, then tried giving it my own ReadOptions and also got the same CSV parse error.

Hi All
I am trying to load a custom dataset  and I am trying to load a single file to make sure the file is loading correctly:
dataset = load_dataset('csv', data_files=files)
When I run it I get:

Downloading and preparing dataset csv/default-35575a1051604c88 (download: Unknown size, generated: Unknown size, post-processed: Unknown size, total: Unknown size) tocache/huggingface/datasets/csv/default-35575a1051604c88/0.0.0/49187751790fa4d820300fd4d0707896e5b941f1a9c644652645b866716a4ac4...

I am getting this error:
6a4ac4/csv.py in _generate_tables(self, files)
     78     def _generate_tables(self, files):
     79         for i, file in enumerate(files):
---> 80             pa_table = pac.read_csv(
     81                 file,
     82                 read_options=self.config.pa_read_options,

~/anaconda2/envs/nlp/lib/python3.8/site-packages/pyarrow/_csv.pyx in pyarrow._csv.read_csv()

~/anaconda2/envs/nlp/lib/python3.8/site-packages/pyarrow/error.pxi in pyarrow.lib.pyarrow_internal_check_status()

~/anaconda2/envs/nlp/lib/python3.8/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status()

**ArrowInvalid: straddling object straddles two block boundaries (try to increase block size?)**

The size of the file is 3.5 GB. When I try smaller files I do not have an issue. When I load it with 'text' parser I can see all data but it is not what I need.
There is no issue reading the file with pandas. any idea what could be the issue?
When I am running a different CSV I do not get  this line:
 (download: Unknown size, generated: Unknown size, post-processed: Unknown size, total: Unknown size)

Any ideas?


Wikipedia postprocessing

Hi @bminixhofer ! Parsing WikiMedia is notoriously difficult: this processing used [mwparserfromhell](https://github.com/earwig/mwparserfromhell) which is pretty good but not perfect.

As an alternative, you can also use the Wiki40b dataset which was pre-processed using an un-released Google internal tool

Ok, thanks! I'll try the Wiki40b dataset.

If anyone else is concerned about this, `wiki40b` does indeed seem very well cleaned.

Hi, thanks for this library!

Running this code:

```py
import datasets
wikipedia = datasets.load_dataset("wikipedia", "20200501.de")
print(wikipedia['train']['text'][0])
```

I get:

```
mini|Ricardo Flores Magón
mini|Mexikanische Revolutionäre, Magón in der Mitte anführend, gegen die Diktatur von Porfirio Diaz, Ausschnitt des Gemälde „Tierra y Libertad“ von Idelfonso Carrara (?) von 1930.

Ricardo Flores Magón (* 16. September 1874 in San Antonio Eloxochitlán im mexikanischen Bundesstaat Oaxaca; † 22. November 1922 im Bundesgefängnis Leavenworth im US-amerikanischen Bundesstaat Kansas) war als Journalist, Gewerkschafter und Literat ein führender anarchistischer Theoretiker und Aktivist, der die revolutionäre mexikanische Bewegung radikal beeinflusste. Magón war Gründer der Partido Liberal Mexicano und Mitglied der Industrial Workers of the World.

Politische Biografie
Journalistisch und politisch kämpfte er und sein Bruder sehr kompromisslos gegen die Diktatur Porfirio Diaz. Philosophisch und politisch orientiert an radikal anarchistischen Idealen und den Erfahrungen seiner indigenen Vorfahren bei der gemeinschaftlichen Bewirtschaftung des Gemeindelandes, machte er die Forderung „Land und Freiheit“ (Tierra y Libertad) populär. Besonders Francisco Villa und Emiliano Zapata griffen die Forderung Land und Freiheit auf. Seine Philosophie hatte großen Einfluss auf die Landarbeiter. 1904 floh er in die USA und gründete 1906 die Partido Liberal Mexicano. Im Exil lernte er u. a. Emma Goldman kennen. Er verbrachte die meiste Zeit seines Lebens in Gefängnissen und im Exil und wurde 1918 in den USA wegen „Behinderung der Kriegsanstrengungen“ zu zwanzig Jahren Gefängnis verurteilt. Zu seinem Tod gibt es drei verschiedene Theorien. Offiziell starb er an Herzversagen. Librado Rivera, der die Leiche mit eigenen Augen gesehen hat, geht davon aus, dass Magón von einem Mitgefangenen erdrosselt wurde. Die staatstreue Gewerkschaftszeitung CROM veröffentlichte 1923 einen Beitrag, nachdem Magón von einem Gefängniswärter erschlagen wurde.
mini|Die Brüder Ricardo (links) und Enrique Flores Magón (rechts) vor dem Los Angeles County Jail, 1917

[...]
```

so some Markup like `mini|` is still left. Should I run another parser on this text before feeding it to an ML model or is this a known imperfection of parsing Wiki markup?

Apologies if this has been asked before.


[GEM] add WikiLingua cross-lingual abstractive summarization dataset

dataset request

Requesting to add a new dataset

Hey @yjernite. This is a very interesting dataset. Would love to work on adding it but I see that the link to the data is to a gdrive folder. Can I just confirm wether dlmanager can handle gdrive urls or would this have to be a manual dl?

Hi @KMFODA ! A version of WikiLingua is actually already accessible in the [GEM dataset](https://huggingface.co/datasets/gem)

You can use it for example to load the French to English translation with:
```python
from datasets import load_dataset
wikilingua = load_dataset("gem", "wiki_lingua_french_fr")
```

Closed by https://github.com/huggingface/datasets/pull/1807

## Adding a Dataset
- **Name:** WikiLingua
- **Description:** The dataset includes ~770k article and summary pairs in 18 languages from WikiHow. The gold-standard article-summary alignments across languages were extracted by aligning the images that are used to describe each how-to step in an article.
- **Paper:** https://arxiv.org/pdf/2010.03093.pdf
- **Data:** https://github.com/esdurmus/Wikilingua
- **Motivation:** Included in the GEM shared task. Multilingual.

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


[GEM] add ASSET text simplification dataset

Requesting to add a new dataset

## Adding a Dataset
- **Name:** ASSET
- **Description:** ASSET is a crowdsourced
multi-reference corpus for assessing sentence simplification in English where each simplification was produced by executing several rewriting transformations.
- **Paper:** https://www.aclweb.org/anthology/2020.acl-main.424.pdf
- **Data:** https://github.com/facebookresearch/asset
- **Motivation:** Included in the GEM shared task

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


[GEM] add WikiAuto text simplification dataset

Requesting to add a new dataset

## Adding a Dataset
- **Name:** WikiAuto
- **Description:** Sentences in English Wikipedia and their corresponding sentences in Simple English Wikipedia that are written with simpler grammar and word choices. A lot of lexical and syntactic paraphrasing.
- **Paper:** https://www.aclweb.org/anthology/2020.acl-main.709.pdf
- **Data:** https://github.com/chaojiang06/wiki-auto
- **Motivation:** Included in the GEM shared task

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


[GEM] Add WebNLG dataset

Requesting to add a new dataset

## Adding a Dataset
- **Name:** WebNLG
- **Description:** WebNLG consists of Data/Text pairs where the data is a set of triples extracted from DBpedia and the text is a verbalisation of these triples (16,095 data inputs and 42,873 data-text pairs). The data is available in English and Russian
- **Paper:** https://www.aclweb.org/anthology/P17-1017.pdf
- **Data:** https://webnlg-challenge.loria.fr/download/
- **Motivation:** Included in the GEM shared task, multilingual

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


[GEM] add ToTTo Table-to-text dataset

Requesting to add a new dataset

## Adding a Dataset
- **Name:** ToTTo
- **Description:** ToTTo is an open-domain English table-to-text dataset with over 120,000 training examples that proposes a controlled generation task: given a Wikipedia table and a set of highlighted table cells, produce a one-sentence description.
- **Paper:** https://arxiv.org/abs/2004.14373
- **Data:** https://github.com/google-research-datasets/totto
- **Motivation:** Included in the GEM shared task

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


[GEM] add Schema-Guided Dialogue

Requesting to add a new dataset

## Adding a Dataset
- **Name:** The Schema-Guided Dialogue Dataset
- **Description:** The Schema-Guided Dialogue (SGD) dataset consists of over 20k annotated multi-domain, task-oriented conversations between a human and a virtual assistant. These conversations involve interactions with services and APIs spanning 20 domains, ranging from banks and events to media, calendar, travel, and weather.
- **Paper:** https://arxiv.org/pdf/2002.01359.pdf https://arxiv.org/pdf/2004.15006.pdf
- **Data:** https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
- **Motivation:** Included in the GEM shared task

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


Add writer_batch_size attribute to GeneratorBasedBuilder

As specified in #741 one would need to specify a custom ArrowWriter batch size to avoid filling the RAM. Indeed the defaults buffer size is 10 000 examples but for multimodal datasets that contain images or videos we may want to reduce that.


[GEM] MultiWOZ dialogue dataset

Requesting to add a new dataset

Hi @yjernite can I help in adding this dataset?

I am excited about this because this will be my first contribution to the datasets library as well as to hugginface.

## Adding a Dataset
- **Name:** MultiWOZ (Multi-Domain Wizard-of-Oz)
- **Description:** 10k annotated human-human dialogues. Each dialogue consists of a goal, multiple user and system utterances as well as a belief state. Only system utterances are annotated with dialogue acts – there are no annotations from the user side.
- **Paper:** https://arxiv.org/pdf/2007.12720.pdf
- **Data:** https://github.com/budzianowski/multiwoz
- **Motivation:** Will likely be part of the GEM shared task

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


[GEM] Add E2E dataset

Requesting to add a new dataset

## Adding a Dataset
- **Name:** E2E NLG dataset (for End-to-end natural language generation)
- **Description:**a dataset for training end-to-end, datadriven natural language generation systems in the restaurant domain, the datasets consists of 5,751 dialogue-act Meaning Representations (structured data) and 8.1 reference free-text utterances per dialogue-act on average
- **Paper:** https://arxiv.org/pdf/1706.09254.pdf https://arxiv.org/abs/1901.07931
- **Data:** http://www.macs.hw.ac.uk/InteractionLab/E2E/#data
- **Motivation:** This dataset will likely be included in the GEM shared task

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


Add accuracy, precision, recall and F1 metrics

This PR adds several single metrics, namely:

- Accuracy
- Precision
- Recall
- F1

They all uses under the hood the sklearn metrics of the same name. They allow different useful features when training a multilabel/multiclass model:
- have a macro/micro/per label/weighted/binary/per sample score
- score only the selected labels (usually what we call the positive labels) and ignore the negative ones. For example in case of a Named Entity Recognition task, positive labels are (`PERSON`, `LOCATION` or `ORGANIZATION`) and the negative one is `O`.


Discussion using datasets in offline mode

enhancement

New feature or request

generic discussion

Generic discussion on the library

I think it would be very cool. I'm currently working on a cluster from Compute Canada, and I have internet access only when I'm not in the nodes where I run the scripts. So I was expecting to be able to use the wmt14 dataset until I realized I needed internet connection even if I downloaded the data already. I'm going to try option 2 you mention for now though! Thanks ;)

Requiring online connection is a deal breaker in some cases unfortunately so it'd be great if offline mode is added similar to how `transformers` loads models offline fine.

@mandubian's second bullet point suggests that there's a workaround allowing you to use your offline (custom?) dataset with `datasets`. Could you please elaborate on how that should look like?

here is my way to load a dataset offline, but it **requires** an online machine
1. (online machine)
```
import datasets
data = datasets.load_dataset(...)
data.save_to_disk(/YOUR/DATASET/DIR)
```
2. copy the dir from online to the offline machine
3. (offline machine)
```
import datasets
data = datasets.load_from_disk(/SAVED/DATA/DIR)
```

HTH.

> here is my way to load a dataset offline, but it **requires** an online machine
>
> 1. (online machine)
>
> ```
>
> import datasets
>
> data = datasets.load_dataset(...)
>
> data.save_to_disk(/YOUR/DATASET/DIR)
>
> ```
>
> 2. copy the dir from online to the offline machine
>
> 3. (offline machine)
>
> ```
>
> import datasets
>
> data = datasets.load_from_disk(/SAVED/DATA/DIR)
>
> ```
>
>
>
> HTH.

I opened a PR that allows to reload modules that have already been loaded once even if there's no internet.

Let me know if you know other ways that can make the offline mode experience better. I'd be happy to add them :)

I already note the "freeze" modules option, to prevent local modules updates. It would be a cool feature.

----------

> @mandubian's second bullet point suggests that there's a workaround allowing you to use your offline (custom?) dataset with `datasets`. Could you please elaborate on how that should look like?

Indeed `load_dataset` allows to load remote dataset script (squad, glue, etc.) but also you own local ones.
For example if you have a dataset script at `./my_dataset/my_dataset.py` then you can do
```python
load_dataset("./my_dataset")
```
and the dataset script will generate your dataset once and for all.

----------

About I'm looking into having `csv`, `json`, `text`, `pandas` dataset builders already included in the `datasets` package, so that they are available offline by default, as opposed to the other datasets that require the script to be downloaded.
cf #1724

The local dataset builders (csv, text , json and pandas) are now part of the `datasets` package since #1726 :)
You can now use them offline
```python
datasets = load_dataset('text', data_files=data_files)
```

We'll do a new release soon

`datasets.load_dataset("csv", ...)` breaks if you have no connection (There is already this issue https://github.com/huggingface/datasets/issues/761 about it). It seems to be the same for metrics too.

I create this ticket to discuss a bit and gather what you have in mind or other propositions.

Here are some points to open discussion:
- if you want to prepare your code/datasets on your machine (having internet connexion) but run it on another offline machine (not having internet connexion), it won't work as is, even if you have all files locally on this machine.
- AFAIK, you can make it work if you manually put the python files (csv.py for example) on this offline machine and change your code to `datasets.load_dataset("MY_PATH/csv.py", ...)`. But it would be much better if you could run ths same code without modification if files are available locally.
- I've also been considering the requirement of downloading Python code and execute on your machine to use datasets. This can be an issue in a professional context. Downloading a CSV/H5 file is acceptable, downloading an executable script can open many security issues. We certainly need a mechanism to at least "freeze" the dataset code you retrieved once so that you can review it if you want and then be sure you use this one everywhere and not a version dowloaded from internet.

WDYT? (thks)


how processing in batch works in datasets

dataset request

Requesting to add a new dataset

Hi I don’t think this is a request for a dataset like you labeled it.

I also think this would be better suited for the forum at https://discuss.huggingface.co. we try to keep the issue for the repo for bug reports and new features/dataset requests and have usage questions discussed on the forum. Thanks.

Hi Thomas,
what I do not get from documentation is that why when you set batched=True,
this is processed in batch, while data is not divided to batched
beforehand, basically this is a question on the documentation and I do not
get the batched=True, but sure, if you think this is more appropriate in
forum I will post it there.
thanks
Best
Rabeeh

On Tue, Nov 10, 2020 at 12:21 PM Thomas Wolf <notifications@github.com>
wrote:

> Hi I don’t think this is a request for a dataset like you labeled it.
>
> I also think this would be better suited for the forum at
> https://discuss.huggingface.co. we try to keep the issue for the repo for
> bug reports and new features/dataset requests and have usage questions
> discussed on the forum. Thanks.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/823#issuecomment-724639476>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ARPXHH4FIPFHVVUHANAE4F3SPEO2JANCNFSM4TQQVEXQ>
> .
>

Yes the forum is perfect for that. You can post in the `datasets` section.
Thanks a lot!

Hi,
I need to process my datasets before it is passed to dataloader in batch,
here is my codes

```
class AbstractTask(ABC):
    task_name: str = NotImplemented
    preprocessor: Callable = NotImplemented
    split_to_data_split: Mapping[str, str] = NotImplemented
    tokenizer: Callable = NotImplemented
    max_source_length: str = NotImplemented
    max_target_length: str = NotImplemented
    # TODO: should not be a task item, but cannot see other ways.
    tpu_num_cores: int = None

    # The arguments set are for all tasks and needs to be kept common.
    def __init__(self, config):
        self.max_source_length = config['max_source_length']
        self.max_target_length = config['max_target_length']
        self.tokenizer = config['tokenizer']
        self.tpu_num_cores = config['tpu_num_cores']

    def _encode(self, batch) -> Dict[str, torch.Tensor]:
        batch_encoding = self.tokenizer.prepare_seq2seq_batch(
            [x["src_texts"] for x in batch],
            tgt_texts=[x["tgt_texts"] for x in batch],
            max_length=self.max_source_length,
            max_target_length=self.max_target_length,
            padding="max_length" if self.tpu_num_cores is not None else "longest",  # TPU hack
            return_tensors="pt"
        )
        return batch_encoding.data

    def data_split(self, split):
        return self.split_to_data_split[split]

    def get_dataset(self, split, n_obs=None):
        split = self.data_split(split)
        if n_obs is not None:
            split = split+"[:{}]".format(n_obs)
        dataset = load_dataset(self.task_name, split=split)
        dataset = dataset.map(self.preprocessor, remove_columns=dataset.column_names)
        dataset = dataset.map(lambda batch: self._encode(batch), batched=True)
        dataset.set_format(type="torch", columns=['input_ids', 'token_type_ids', 'attention_mask', 'label'])
        return dataset

```

I call it like

`AutoTask.get(task, train_dataset_config).get_dataset(split="train", n_obs=data_args.n_train)
`

This gives the following error, to me because the data inside the   dataset = dataset.map(lambda batch: self._encode(batch), batched=True) is not processed in batch, could you tell me how I can process dataset in batch inside my function? thanks

  File "finetune_multitask_trainer.py", line 192, in main
    if training_args.do_train else None
  File "finetune_multitask_trainer.py", line 191, in <dictcomp>
    split="train", n_obs=data_args.n_train) for task in data_args.task}
  File "/remote/idiap.svm/user.active/rkarimi/dev/internship/seq2seq/tasks.py", line 56, in get_dataset
    dataset = dataset.map(lambda batch: self._encode(batch), batched=True)
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1236, in map
    update_data = does_function_return_dict(test_inputs, test_indices)
  File "/idiap/user/rkarimi/libs/anaconda3/envs/internship/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1207, in does_function_return_dict
    function(*fn_args, indices, **fn_kwargs) if with_indices else function(*fn_args, **fn_kwargs)
  File "/remote/idiap.svm/user.active/rkarimi/dev/internship/seq2seq/tasks.py", line 56, in <lambda>
    dataset = dataset.map(lambda batch: self._encode(batch), batched=True)
  File "/remote/idiap.svm/user.active/rkarimi/dev/internship/seq2seq/tasks.py", line 37, in _encode
    [x["src_texts"] for x in batch],
  File "/remote/idiap.svm/user.active/rkarimi/dev/internship/seq2seq/tasks.py", line 37, in <listcomp>
    [x["src_texts"] for x in batch],
TypeError: string indices must be integers


datasets freezes

Pytorch is unable to convert strings to tensors unfortunately.
You can use `set_format(type="torch")` on columns that can be converted to tensors, such as token ids.

This makes me think that we should probably raise an error or at least a warning when one tries to create pytorch tensors out of text columns

Hi, I want to load these two datasets and convert them to Dataset format in torch and the code freezes for me, could you have a look please? thanks

dataset1 = load_dataset("squad", split="train[:10]")
dataset1 = dataset1.set_format(type='torch', columns=['context', 'answers', 'question'])

dataset2 = load_dataset("imdb", split="train[:10]")
dataset2 = dataset2.set_format(type="torch", columns=["text", "label"])
print(len(dataset1))


`kor_nli` dataset doesn't being loaded properly

There are two issues from `kor_nli` dataset

1. csv.DictReader failed to split features by tab
    - Should not exist `None` value in label feature, but there it is.
        ```python
        kor_nli_train['train'].unique('gold_label')
        # ['neutral', 'entailment', 'contradiction', None]
        ```
    - I found a reason why there is `None` values in label feature as following code
        ```python
        from datasets import load_dataset
        kor_nli_train = load_dataset('kor_nli', 'multi_nli')

        for idx, example in enumerate(kor_nli_train['train']):
            if example['gold_label'] is None:
                print(idx, example)
                break
        # 16835 {'gold_label': None, 'sentence1': '그는 전쟁 전에 가벼운 벅스킨 암말을 가지고 달리기 위해 우유처럼 하얀 스터드를 넣었다.\t전쟁 전에 다인종 여성들과 함께 있는 백인 남자가 있었다.\tentailment\n슬림은 재빨리 옷을 입었고, 순간적으로 미지근한 물을 뿌릴 수 있는 아침 세탁물을 기꺼이 가두었다.\t슬림은 직장에 늦었다.\tneutral\n뉴욕에서 그 식사를 해봤는데, 거기서 소고기의 멋진 소고기 부분을 요리하고 바베큐로 만든 널빤지 같은 걸 가져왔는데, 정말 대단해.\t그들이 거기서 요리하는 쇠고기는 역겹다. 거기서 절대 먹지 마라.\tcontradiction\n판매원의 죽음에서 브라이언 데네히... 크리스 켈리\t크리스 켈리는 세일즈맨의 죽음을 언급하지 않는다.\tcontradiction\n그러는 동안 요리사는 그냥 화가 났어.\t스튜가 끓는 동안 요리사는 화가 났다.\tneutral\n마지막 로마의 맹공격 전날 밤, 900명 이상의 유대인 수비수들이 로마인들에게 그들을 사로잡는 승리를 주기 보다는 대량 자살을 저질렀다.\t로마인들이 그들의 포획에 승리하도록 내버려두기 보다는 900명의 유대인 수비수들이 자살했다.\tentailment\n앞으로 발사하라.\t발사.\tneutral\n그리고 당신은 우리 땅이 에이커에 있다는 것을 알고 있다. 우리 사람들은 어떤 것이 얼마나 많은지 이해하지 못할 것이다.\t모든 사람들은 우리의 측정 시스템이 어떻게 작동하는지 알고 이해합니다.\tcontradiction\n주미게스\tJumiyges는 도시의 이름이다.\tneutral\n사람은 자기 민족을 돌봐야 한다...\t사람은 조국에 공감해야 한다.\tentailment\n또한 PDD 63은 정부와 업계가 컴퓨터 기반 공격에 대해 경고하고 방어할 준비를 더 잘할 수 있도록 시스템 취약성, 위협, 침입 및 이상에 대한 정보를 공유하는 메커니즘을 수립하는 것이 중요하다는 것을 인식했습니다.\t정보 전송 프로토콜을 만드는 것은 중요하다.\tentailment\n카페 링 피아자 델라 레퓌블리카 바로 남쪽에는 피렌체가 알려진 짚 제품 때문에 한때 스트로 마켓이라고 불렸던 16세기 로지아인 메르카토 누오보(Mercato Nuovo)가 있다.\t피아자 델라 레퓌블리카에는 카페가 많이 있다.\tentailment\n우리가 여기 있는 한 트린판이 뭘 주웠는지 살펴봐야겠어\t우리는 트린판이 무엇을 주웠는지 보는 데 시간을 낭비하지 않을 것이다.\tcontradiction\n그러나 켈트족의 문화적 기반을 가진 아일랜드 교회는 유럽의 신흥 기독교 세계와는 다르게 발전했고 결국 로마와 중앙집권적 행정으로 대체되었다.\t아일랜드 교회에는 켈트족의 기지가 있었다.\tentailment\n글쎄, 넌 선택의 여지가 없어\t글쎄, 너에겐 많은 선택권이 있어.\tcontradiction\n사실, 공식적인 보장은 없다.\t내가 산 물건에 대한 보증이 없었다.\tneutral\n덜 활기차긴 하지만, 안시와 르 부르젯의 사랑스러운 호수에서도 삶은 똑같이 상쾌하다.\t안시와 르 부르겟에서는 호수에서의 활동이 서두르고 바쁜 분위기를 연출한다.\tcontradiction\n그의 여행 소식이 이미 퍼졌다면 공격 소식도 퍼졌을 테지만 마을에서는 전혀 공황의 기미가 보이지 않았다.\t그는 왜 마을이 당황하지 않았는지 알 수 없었다.\tneutral\n과거에는 죽음의 위협이 토지의 판매를 막는 데 거의 도움이 되지 않았다.\t토지 판매는 어떠한 위협도 교환하지 않고 이루어진다.\tcontradiction\n어느 시점에 이르러 나는 지금 다가오는 새로운 것들과 나오는 많은 새로운 것들이 내가 늙어가고 있다고 말하는 시대로 접어들고 있다.\t나는 여전히 내가 보는 모든 새로운 것을 사랑한다.\tcontradiction\n뉴스위크는 물리학자들이 경기장 행사에서 고속도로의 자동차 교통과 보행자 교통을 개선하기 위해 새떼의 움직임을 연구하고 있다고 말한다.\t고속도로의 자동차 교통 흐름을 개선하는 것은 물리학자들이 새떼를 연구하는 이유 중 하나이다.\tentailment\n얼마나 다른가? 그는 잠시 말을 멈추었다가 말을 이었다.\t그는 그 소녀가 어디에 있는지 알고 싶었다.\tentailment\n글쎄, 그에게 너무 많은 것을 주지마.\t그는 훨씬 더 많은 것을 요구할 것이다.\tneutral\n아무리 그의 창작물이 완벽해 보인다고 해도, 그들을 믿는 것은 아마도 좋은 생각이 아닐 것이다.\'\t도자기를 잘 만든다고 해서 누군가를 믿는 것은 아마 좋지 않을 것이다.\tneutral\n버스틀링 그란 비아(Bustling Gran Via)는 호텔, 상점, 극장, 나이트클럽, 카페 등이 어우러져 산책과 창가를 볼 수 있다.\tGran Via는 호텔, 상점, 극장, 나이트클럽, 카페의 번화한 조합이다.\tentailment\n정부 인쇄소\t그 사무실은 워싱턴에 위치해 있다.\tneutral\n실제 문화 전쟁이 어디 있는지 알고 싶다면 학원을 잊어버리고 실리콘 밸리와 레드몬드를 생각해 보라.\t실제 문화 전쟁은 레드몬드에서 일어난다.\tentailment\n그리고 페니실린을 주지 않기 위해 침대 위에 올려놨어\t그녀의 방에는 페니실린이 없다는 징후가 전혀 없었다.\tcontradiction\nL.A.의 야외 시장을 활보하는 것은 맛있고 저렴한 그루브를 잡고, 끝이 없는 햇빛을 즐기고, 신선한 농산물, 꽃, 향, 그리고 가젯 갈로어를 구입하면서 현지인들과 어울릴 수 있는 훌륭한 방법이다.\tLA의 야외 시장을 돌아다니는 것은 시간 낭비다.\tcontradiction\n안나는 밖으로 나와 안도의 한숨을 내쉬었다. 단 한 번, 그리고 마리후아쉬 맛의 술로 끝내자는 결심이 뒤섞여 있었다.\t안나는 안심하고 마리후아쉬 맛의 술을 다 마시기로 결심했다.\tentailment\n5 월에 Vajpayee는 핵 실험의 성공적인 완료를 발표했는데, 인도인들은 주권의 표시로 선전했지만 이웃 국가와 서구와의 인도 관계를 복잡하게 만들 수 있습니다.\t인도는 성공적인 핵실험을 한 적이 없다.\tcontradiction\n플라노 원에서 보통 얼마나 많은 것을 가지고 있는가?\t저 사람들 중에 플라노 원에 가본 사람 있어?\tcontradiction\n그것의 전체적인 형태의 우아함은 운하 건너편에서 가장 잘 볼 수 있다. 왜냐하면, 로마에 있는 성 베드로처럼, 돔은 길쭉한 본당 뒤로 더 가까운 곳에 사라지기 때문이다.\t성 베드로의 길쭉한 본당은 돔을 가린다.\tentailment\n당신은 수틴이 살에 강박적인 기쁨을 가지고 누드를 그릴 것이라고 생각하겠지만, 아니오; 그는 그의 모든 경력에서 단 한 점만을 그렸고, 그것은 사소한 그림이다.\t그는 그것이 그를 불편하게 만들었기 때문에 하나만 그렸다.\tneutral\n이 인상적인 풍경은 원래 나포 레온이 루브르 박물관의 침실에서 볼 수 있도록 계획되었는데, 그 당시 궁전이었습니다.\t나폴레옹은 그의 모든 궁전에 있는 그의 침실에서 보는 경치에 많은 관심을 가졌다.\tneutral\n그는 우리에게 문 열쇠를 건네주고는 급히 떠났다.\t그는 긴장해서 우리에게 열쇠를 빨리 주었다.\tneutral\n위원회는 또한 최종 규칙을 OMB에 제출했다.\t위원회는 또한 이 규칙을 다른 그룹에 제출했지만 최종 규칙은 OMB가 평가하기 위한 것이 었습니다.\tneutral\n정원가게에 가보면 올리비아의 복제 화합물 같은 유쾌한 이름을 가진 제품들을 찾을 수 있을 겁니다.이 제품이 뿌리를 내리도록 돕기 위해 촬영의 절단된 끝에 덩크슛을 하는 호르몬의 혼합물이죠.\t정원 가꾸기 가게의 제품들은 종종 그들의 목적을 설명하기 위해 기술적으로나 과학적으로 파생된 이름(올리비아의 복제 화합물처럼)을 부여받는다.\tneutral\n스타는 스틸 자신이나 왜 그녀의 이야기를 바꾸었는지에 훨씬 더 관심이 있을 것이다.\t스틸의 이야기는 조금도 변하지 않았다.\tcontradiction\n남편과의 마지막 대결로 맥티어는 노라의 변신을 너무나 능숙하게 예고해 왔기 때문에, 그녀에게는 당황스러울 정도로 갑작스러운 것처럼 보이지만, 우리에게는 감정적으로 불가피해 보인다.\t노라의 변신은 분명하고 필연적이었다.\tcontradiction\n이집트 최남단 도시인 아스완은 오랜 역사를 통해 중요한 역할을 해왔다.\t아스완은 이집트 국경 바로 위에 위치해 있습니다.\tneutral\n그러나 훨씬 더 우아한 건축적 터치는 신성한 춤인 Bharatanatyam에서 수행된 108 가지 기본 포즈를 시바 패널에서 볼 수 있습니다.\t패널에 대한 시바의 묘사는 일반적인 모티브다.\tneutral\n호화롭게 심어진 계단식 정원은 이탈리아 형식의 가장 훌륭한 앙상블 중 하나입니다.\t아름다운 정원과 희귀한 꽃꽂이 모두 이탈리아의 형식적인 스타일을 보여준다.\tneutral\n음, 그랬으면 좋았을 텐데\t나는 그것을 다르게 할 기회를 몹시 갈망한다.\tentailment\n폐허가 된 성의 기슭에 자리잡고 있는 예쁜 중세 도시 케이서스버그는 노벨 평화상 수상자 알버트 슈바이처(1875년)의 출생지로 널리 알려져 있다.\t알버트 슈바이처는 둘 다 케이서스버그 마을에 있었다.\tentailment\n고감도는 문제가 있는 대부분의 환자들이 발견될 것을 보장한다.\t장비 민감도는 문제 탐지와 관련이 없습니다.\tcontradiction\n오늘은 확실히 반바지 같은 날이었어\t오늘 사무실에 있는 모든 사람들은 반바지를 입었다.\tneutral\n못생긴 턱시도를 입고.\t그것은 분홍색과 주황색입니다.\tneutral\n이주 노동 수용소 오 마이 갓 그들은 판지 상자에 산다.\t노동 수용소에는 판지 상자에 사는 이주 노동자들의 사진이 있다.\tneutral\n그래, 그가 전 세계를 여행한 후에 그런 거야\t그것은 사람들의 세계 여행을 따른다.\tentailment\n건너편에 크고 큰 참나무 몇 그루가 있다.\t우리는 여기 오크나 어떤 종류의 미국 나무도 없다.\tcontradiction\nFort-de-France에서 출발하는 자동차나 여객선으로, 당신은 안세 ? 바다 포도가 그늘을 제공하는 쾌적한 갈색 모래 해변과 피크닉 테이블, 어린이 미끄럼틀, 식당이 있는 안느에 도착할 수 있다.\t프랑스 요새에서 자동차나 페리를 타고 안세로 갈 수 있다.\tentailment\n그리고 그것은 앨라배마주가 예상했던 대로 예산에서 50만 달러를 삭감하지 않을 것이라는 것을 의미한다.\t앨라배마 주는 예산 삭감을 하지 않았다. 왜냐하면 그렇게 하는 것에 대한 초기 정당성이 정밀 조사에 맞서지 않았기 때문이다.\tneutral\n알았어 먼저 어 .. 어 .. 노인이나 가족을 요양원에 보내는 것에 대해 어떻게 생각하니?\t가족을 요양원에 보내서 사는 것에 대해 어떻게 생각하는지 알 필요가 없다.\tcontradiction\n나머지는 너에게 달렸어.\t나머지는 너에게 달렸지만 시간이 많지 않다.\tneutral\n음-흠, 3월에 햇볕에 타는 것에 대해 걱정하면 안 된다는 것을 알고 있는 3월이야.\t3월은 그렇게 덥지 않다.\tneutral\n그리고 어, 그런 작은 것들로 다시 시작해봐. 아직 훨씬 싸. 어, 그 특별한 모델 차는 150달러야.\t그 모형차는 4천 달러가 든다.\tcontradiction\n내일 돌아가야 한다면, 칼이 말했다.\t돌아갈 수 없어. 오늘은 안 돼. 내일은 안 돼. 절대 안 돼." 칼이 말했다.', 'sentence2': 'contradiction'}
        ```

2. (Optional) Preferred to change the name of the features for the compatibility with `run_glue.py` in 🤗 Transformers
    - `kor_nli` dataset has same data structure of multi_nli, xnli
    - Changing the name of features and the feature type of 'gold_label' to ClassLabel might be helpful
    ```python
        def _info(self):
            return datasets.DatasetInfo(
                description=_DESCRIPTION,
                features=datasets.Features(
                    {
                        "premise": datasets.Value("string"),
                        "hypothesis": datasets.Value("string"),
                        "label": datasets.features.ClassLabel(names=["entailment", "neutral", "contradiction"]),
                    }
                ),
    ```

If you don't mind, I would like to fix this.
Thanks!


Update quail dataset to v1.3

Updated quail to most recent version, to address the problem originally discussed [here](https://github.com/huggingface/datasets/issues/806).


Make save function use deterministic global vars order

The `dumps` function need to be deterministic for the caching mechanism.
However in #816 I noticed that one of dill's method to recursively check the globals of a function may return the globals in different orders each time it's used. To fix that I sort the globals by key in the `globs` dictionary.
I had to add a rectified `save_function` to the saving functions registry of the Pickler to make it work.

This should fix #816


Fix type hints pickling in python 3.6

Type hints can't be properly pickled in python 3.6. This was causing errors the `run_mlm.py` script from `transformers` with python 3.6

However Cloupickle proposed a [fix](https://github.com/cloudpipe/cloudpickle/pull/318/files) to make it work anyway.
The idea is just to implement the pickling/unpickling of parameterized type hints. There is one detail though: since in python 3.6 we can't use `isinstance` on type hints, then we can't use pickle saving functions registry directly. Therefore we just wrap the `save_global` method of the Pickler.

This should fix https://github.com/huggingface/transformers/issues/8212 for python 3.6 and make `run_mlm.py` support python 3.6

cc @sgugger


Add MRQA dataset

Requesting to add a new dataset

## Adding a Dataset
- **Name:** MRQA
- **Description:** Collection of different (subsets of) QA datasets all converted to the same format to evaluate out-of-domain generalization (the datasets come from different domains, distributions, etc.). Some datasets are used for training and others are used for evaluation. This dataset was collected as part of MRQA 2019's shared task
- **Paper:** https://arxiv.org/abs/1910.09753
- **Data:** https://github.com/mrqa/MRQA-Shared-Task-2019
- **Motivation:** Out-of-domain generalization is becoming (has become) a de-factor evaluation for NLU systems

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


[Caching] Dill globalvars() output order is not deterministic and can cause cache issues.

To show the issue:
```
python -c "from datasets.fingerprint import Hasher; a=[]; func = lambda : len(a); print(Hasher.hash(func))"
```
doesn't always return the same ouput since `globs` is a dictionary with "a" and "len" as keys but sometimes not in the same order

Dill uses `dill.detect.globalvars` to get the globals used by a function in a recursive dump. `globalvars` returns a dictionary of all the globals that a dumped function needs. However the order of the keys in this dict is not deterministic and can cause caching issues.

To fix that one could register an implementation of dill's `save_function` in the `datasets` pickler that sorts the globals keys before dumping a function.


Is dataset iterative or not?

Requesting to add a new dataset

Hello !
Could you give more details ?

If you mean iter through one dataset then yes, `Dataset` object does implement the `__iter__` method so you can use
```python
for example in dataset:
    # do something
```

If you want to iter through several datasets you can first concatenate them
```python
from datasets import concatenate_datasets

new_dataset = concatenate_datasets([dataset1, dataset2])
```
Let me know if this helps !

Hi Huggingface/Datasets team,
I want to use the datasets inside Seq2SeqDataset here
https://github.com/huggingface/transformers/blob/master/examples/seq2seq/utils.py
and there I need to return back each line from the datasets and I am not
sure how to access each line and implement this?
It seems it also has get_item attribute? so I was not sure if this is
iterative dataset? or if this is non-iterable datasets?
thanks.

On Mon, Nov 9, 2020 at 10:18 AM Quentin Lhoest <notifications@github.com>
wrote:

> Hello !
> Could you give more details ?
>
> If you mean iter through one dataset then yes, Dataset object does
> implement the __iter__ method so you can use
>
> for example in dataset:
>     # do something
>
> If you want to iter through several datasets you can first concatenate them
>
> from datasets import concatenate_datasets
> new_dataset = concatenate_datasets([dataset1, dataset2])
>
> Let me know if this helps !
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/815#issuecomment-723881199>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ARPXHHYRLSSYW6NZN2HYDBTSO6XV5ANCNFSM4TPB7OWA>
> .
>

could you tell me please if datasets also has __getitem__ any idea on how
to integrate it with Seq2SeqDataset is appreciated thanks

On Mon, Nov 9, 2020 at 10:22 AM Rabeeh Karimi Mahabadi <rabeeh@google.com>
wrote:

> Hi Huggingface/Datasets team,
> I want to use the datasets inside Seq2SeqDataset here
> https://github.com/huggingface/transformers/blob/master/examples/seq2seq/utils.py
> and there I need to return back each line from the datasets and I am not
> sure how to access each line and implement this?
> It seems it also has get_item attribute? so I was not sure if this is
> iterative dataset? or if this is non-iterable datasets?
> thanks.
>
>
>
> On Mon, Nov 9, 2020 at 10:18 AM Quentin Lhoest <notifications@github.com>
> wrote:
>
>> Hello !
>> Could you give more details ?
>>
>> If you mean iter through one dataset then yes, Dataset object does
>> implement the __iter__ method so you can use
>>
>> for example in dataset:
>>     # do something
>>
>> If you want to iter through several datasets you can first concatenate
>> them
>>
>> from datasets import concatenate_datasets
>> new_dataset = concatenate_datasets([dataset1, dataset2])
>>
>> Let me know if this helps !
>>
>> —
>> You are receiving this because you authored the thread.
>> Reply to this email directly, view it on GitHub
>> <https://github.com/huggingface/datasets/issues/815#issuecomment-723881199>,
>> or unsubscribe
>> <https://github.com/notifications/unsubscribe-auth/ARPXHHYRLSSYW6NZN2HYDBTSO6XV5ANCNFSM4TPB7OWA>
>> .
>>
>

`datasets.Dataset` objects implement indeed `__getitem__`. It returns a dictionary with one field per column.

We've not added the integration of the datasets library for the seq2seq utilities yet. The current seq2seq utilities are based on text files.

However as soon as you have a `datasets.Dataset` with columns "tgt_texts" (str), "src_texts" (str), and "id" (int) you should be able to implement your own Seq2SeqDataset class that wraps your dataset object. Does that make sense to you ?

Hi
I am sorry for asking it multiple times but I am not getting the dataloader
type, could you confirm if the dataset library returns back an iterable
type dataloader or a mapping type one where one has access to __getitem__,
in the former case, one can iterate with __iter__, and how I can configure
it to return the data back as the iterative type? I am dealing with
large-scale datasets and  I do not want to bring all in memory
thanks for your help
Best regards
Rabeeh

On Mon, Nov 9, 2020 at 11:17 AM Quentin Lhoest <notifications@github.com>
wrote:

> datasets.Dataset objects implement indeed __getitem__. It returns a
> dictionary with one field per column.
>
> We've not added the integration of the datasets library for the seq2seq
> utilities yet. The current seq2seq utilities are based on text files.
>
> However as soon as you have a datasets.Dataset with columns "tgt_texts"
> (str), "src_texts" (str), and "id" (int) you should be able to implement
> your own Seq2SeqDataset class that wraps your dataset object. Does that
> make sense ?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/815#issuecomment-723915556>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ARPXHHYOC22EM7F666BZSOTSO66R3ANCNFSM4TPB7OWA>
> .
>

`datasets.Dataset` objects are both iterative and mapping types: it has both `__iter__` and `__getitem__`
For example you can do
```python
for example in dataset:
    # do something
```
or
```python
for i in range(len(dataset)):
    example = dataset[i]
    # do something
```
When you do that, one and only one example is loaded into memory at a time.

Hi there,
Here is what I am trying, this is not working for me in map-style datasets, could you please tell me how to use datasets with being able to access ___getitem__ ? could you assist me please correcting this example? I need map-style datasets which is formed from concatenation of two datasets from your library. thanks

```
import datasets
dataset1 = load_dataset("squad", split="train[:10]")
dataset1 = dataset1.map(lambda example: {"src_texts": "question: {0} context: {1} ".format(
    example["question"], example["context"]),
    "tgt_texts": example["answers"]["text"][0]}, remove_columns=dataset1.column_names)
dataset2 = load_dataset("imdb", split="train[:10]")
dataset2 = dataset2.map(lambda example: {"src_texts": "imdb: " + example["text"],
            "tgt_texts": str(example["label"])}, remove_columns=dataset2.column_names)
train_dataset = datasets.concatenate_datasets([dataset1, dataset2])
train_dataset.set_format(type='torch', columns=['src_texts', 'tgt_texts'])
dataloader = torch.utils.data.DataLoader(train_dataset, batch_size=32)
for id, batch in enumerate(dataloader):
    print(batch)

```

closed since I found this response on the issue https://github.com/huggingface/datasets/issues/469

Hi
I want to use your library for large-scale training, I am not sure if this is implemented as iterative datasets or not?
could you provide me with example how I can use datasets as iterative datasets?
thanks


Joining multiple datasets

Requesting to add a new dataset

found a solution here  https://discuss.pytorch.org/t/train-simultaneously-on-two-datasets/649/35, closed for now, thanks

Hi
I have multiple iterative datasets from your library with different size and I want to join them in a way that each datasets is sampled equally, so smaller datasets more, larger one less, could you tell me how to implement this in pytorch? thanks


How to implement DistributedSampler with datasets

Requesting to add a new dataset

Hi Apparently I need to shard the data and give one host a chunk, could you provide me please with examples on how to do it? I want to use it jointly with finetune_trainer.py in huggingface repo seq2seq examples. thanks.

Hey @rabeehkarimimahabadi I'm actually looking for the same feature. Did you manage to get somewhere?

Hi,
I am using your datasets to define my dataloaders, and I am training finetune_trainer.py in huggingface repo on them.
I need a distributedSampler to be able to train the models on TPUs being able to distribute the load across the TPU cores. Could you tell me how I can implement the distribued sampler when using datasets in which datasets are iterative? To give you more context, I have multiple of datasets and I need to write sampler for this case. thanks.


Too much logging

Hi ! Thanks for reporting :)
I agree these one should be hidden when the logging level is warning, we'll fix that

+1, the amount of logging is excessive.

Most of it indeed comes from `filelock.py`, though there are occasionally messages from other sources too. Below is an example (all of these messages were logged after I already called `datasets.logging.set_verbosity_error()`)

```
I1109 21:26:01.742688 139785006901056 filelock.py:318] Lock 139778216292192 released on /home/kitaev/.cache/huggingface/datasets/9ed4f2e133395826175a892c70611f68522c7bc61a35476e8b51a31afb76e4bf.e6f3e3f3e3875a07469d1cfd32e16e1d06b149616b11eef2d081c43d515b492d.py.lock
I1109 21:26:01.747898 139785006901056 filelock.py:274] Lock 139778216290176 acquired on /home/kitaev/.cache/huggingface/datasets/_home_kitaev_.cache_huggingface_datasets_glue_mnli_1.0.0_7c99657241149a24692c402a5c3f34d4c9f1df5ac2e4c3759fadea38f6cb29c4.lock
I1109 21:26:01.748258 139785006901056 filelock.py:318] Lock 139778216290176 released on /home/kitaev/.cache/huggingface/datasets/_home_kitaev_.cache_huggingface_datasets_glue_mnli_1.0.0_7c99657241149a24692c402a5c3f34d4c9f1df5ac2e4c3759fadea38f6cb29c4.lock
I1109 21:26:01.748412 139785006901056 filelock.py:274] Lock 139778215853024 acquired on /home/kitaev/.cache/huggingface/datasets/_home_kitaev_.cache_huggingface_datasets_glue_mnli_1.0.0_7c99657241149a24692c402a5c3f34d4c9f1df5ac2e4c3759fadea38f6cb29c4.lock
I1109 21:26:01.748497 139785006901056 filelock.py:318] Lock 139778215853024 released on /home/kitaev/.cache/huggingface/datasets/_home_kitaev_.cache_huggingface_datasets_glue_mnli_1.0.0_7c99657241149a24692c402a5c3f34d4c9f1df5ac2e4c3759fadea38f6cb29c4.lock
I1109 21:07:17.029001 140301730502464 filelock.py:274] Lock 140289479304360 acquired on /home/kitaev/.cache/huggingface/datasets/b16d3a04bf2cad1346896852bf120ba846ea1bebb1cd60255bb3a1a2bbcc3a67.ec871b06a00118091ec63eff0a641fddcb8d3c7cd52e855bbb2be28944df4b82.py.lock
I1109 21:07:17.029341 140301730502464 filelock.py:318] Lock 140289479304360 released on /home/kitaev/.cache/huggingface/datasets/b16d3a04bf2cad1346896852bf120ba846ea1bebb1cd60255bb3a1a2bbcc3a67.ec871b06a00118091ec63eff0a641fddcb8d3c7cd52e855bbb2be28944df4b82.py.lock
I1109 21:07:17.058964 140301730502464 filelock.py:274] Lock 140251889388120 acquired on /home/kitaev/.cache/huggingface/metrics/glue/mnli/default_experiment-1-0.arrow.lock
I1109 21:07:17.060933 140301730502464 filelock.py:318] Lock 140251889388120 released on /home/kitaev/.cache/huggingface/metrics/glue/mnli/default_experiment-1-0.arrow.lock
I1109 21:07:17.061067 140301730502464 filelock.py:274] Lock 140296072521488 acquired on /home/kitaev/.cache/huggingface/metrics/glue/mnli/default_experiment-1-0.arrow.lock
I1109 21:07:17.069736 140301730502464 metric.py:400] Removing /home/kitaev/.cache/huggingface/metrics/glue/mnli/default_experiment-1-0.arrow
I1109 21:07:17.069949 140301730502464 filelock.py:318] Lock 140296072521488 released on /home/kitaev/.cache/huggingface/metrics/glue/mnli/default_experiment-1-0.arrow.lock
```

In the latest version of the lib the logs about locks are at the DEBUG level so you won't see them by default.
Also `set_verbosity_warning` does take into account these logs now.
Can you try to update the lib ?
```
pip install --upgrade datasets
```

Thanks. For some reason I have to use the older version. Is that possible I can fix this by some surface-level trick?

I'm still using 1.13 version datasets.

On older versions you can use
```python
import logging

logging.getLogger("filelock").setLevel(logging.WARNING)
```

I'm doing this in the beginning of my script:

from datasets.utils import logging as datasets_logging
datasets_logging.set_verbosity_warning()

but I'm still getting these logs:

[2020-11-07 15:45:41,908][filelock][INFO] - Lock 139958278886176 acquired on /home/username/.cache/huggingface/datasets/cfe20ffaa80ef1c145a0a210d5b9cdce2b60002831e6ed0edc7ab9275d6f0d48.1bd4ccbce9de3dad0698d84674a19d6cc66a84db736a6398110bd196795dde7e.py.lock

[2020-11-07 15:45:41,909][filelock][INFO] - Lock 139958278886176 released on /home/username/.cache/huggingface/datasets/cfe20ffaa80ef1c145a0a210d5b9cdce2b60002831e6ed0edc7ab9275d6f0d48.1bd4ccbce9de3dad0698d84674a19d6cc66a84db736a6398110bd196795dde7e.py.lock

using datasets version = 1.1.2


nlp viewer error

Is this the problem of my local computer or ??

Hello,
when I select amazon_us_reviews in nlp viewer, it shows error.
https://huggingface.co/nlp/viewer/?dataset=amazon_us_reviews
![image](https://user-images.githubusercontent.com/30210529/98447334-4aa81200-2124-11eb-9dca-82c3ab34ccc2.png)


Fix seqeval metric

The current seqeval metric returns the following error when computed:
```
~/.cache/huggingface/modules/datasets_modules/metrics/seqeval/78a944d83252b5a16c9a2e49f057f4c6e02f18cc03349257025a8c9aea6524d8/seqeval.py in _compute(self, predictions, references, suffix)
    102         scores = {}
    103         for type_name, score in report.items():
--> 104             scores[type_name]["precision"] = score["precision"]
    105             scores[type_name]["recall"] = score["recall"]
    106             scores[type_name]["f1"] = score["f1-score"]

KeyError: 'LOC'
```
This is because the current code basically tries to do:
```
scores = {}
scores["LOC"]["precision"] = some_value
```
which does not work in python. This PR fixes that while keeping the previous nested structure of results, with the same keys.


Add Google Taskmaster dataset

Requesting to add a new dataset

Hey @yjernite. Was going to start working on this but found taskmaster 1,2 & 3 in the datasets library already so think this can be closed now?

You are absolutely right :)

Closed by https://github.com/huggingface/datasets/pull/1193 https://github.com/huggingface/datasets/pull/1197 https://github.com/huggingface/datasets/pull/1213

## Adding a Dataset
- **Name:** Taskmaster
- **Description:** A large dataset of task-oriented dialogue with annotated goals (55K dialogues covering entertainment and travel reservations)
- **Paper:** https://arxiv.org/abs/1909.05358
- **Data:** https://github.com/google-research-datasets/Taskmaster
- **Motivation:** One of few annotated datasets of this size for goal-oriented dialogue

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


dataset(dgs): initial dataset loading script

Hi @AmitMY,

Were you able to figure this out?

I did not.
With all the limitations this repo currently has, I had to create a repo of my own using tfds to mitigate them.
https://github.com/sign-language-processing/datasets/tree/master/sign_language_datasets/datasets/dgs_corpus

Closing as I don't know how to support this PR further

When trying to create dummy data I get:

> Dataset datasets with config None seems to already open files in the method `_split_generators(...)`. You might consider to instead only open files in the method `_generate_examples(...)` instead. If this is not possible the dummy data has t o be created with less guidance. Make sure you create the file dummy_data.

I am not sure how to manually create the dummy_data (what exactly it should contain)

Also note, this library says:
> ImportError: To be able to use this dataset, you need to install the following dependencies['pympi'] using 'pip install pympi' for instance'

When you actually need to `pip install pympi-ling`


Hi !
The url works on my side.

Is the url working in your navigator ?
Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?

> Hi !
> The url works on my side.
>
> Is the url working in your navigator ?
> Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?

I tried another server, it's working now. Thanks a lot.

And I'm curious about why download things from "github" when I load dataset from local files ?  Dose datasets work if my network crashed?

It seems my network frequently crashed so most time it cannot work.

> > Hi !
> > The url works on my side.
> > Is the url working in your navigator ?
> > Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?
>
> I tried another server, it's working now. Thanks a lot.
>
> And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?

I download the scripts `https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py` and move it to the package dir `*/datasets/` solved the problem. Could you please put the file `datasets/datasets/csv/csv.py` to `datasets/src/datasets/`？

Thanks :D

hello, how did you solve this problems?

> > > Hi !
> > > The url works on my side.
> > > Is the url working in your navigator ?
> > > Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?
> >
> >
> > I tried another server, it's working now. Thanks a lot.
> > And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?
>
> I download the scripts `https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py` and move it to the package dir `*/datasets/` solved the problem. Could you please put the file `datasets/datasets/csv/csv.py` to `datasets/src/datasets/`？
>
> Thanks :D

hello, I tried this. but it still failed. how do you fix this error?

> hello, how did you solve this problems?
>
> > > > Hi !
> > > > The url works on my side.
> > > > Is the url working in your navigator ?
> > > > Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?
> > >
> > >
> > > I tried another server, it's working now. Thanks a lot.
> > > And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?
> >
> >
> > I download the scripts `https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py` and move it to the package dir `*/datasets/` solved the problem. Could you please put the file `datasets/datasets/csv/csv.py` to `datasets/src/datasets/`？
> > Thanks :D
>
> hello, I tried this. but it still failed. how do you fix this error?

你把那个脚本下载到你本地安装目录下，然后 `load_dataset(csv_script_path, data_fiels)`

> > hello, how did you solve this problems?
> > > > > Hi !
> > > > > The url works on my side.
> > > > > Is the url working in your navigator ?
> > > > > Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?
> > > >
> > > >
> > > > I tried another server, it's working now. Thanks a lot.
> > > > And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?
> > >
> > >
> > > I download the scripts `https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py` and move it to the package dir `*/datasets/` solved the problem. Could you please put the file `datasets/datasets/csv/csv.py` to `datasets/src/datasets/`？
> > > Thanks :D
> >
> >
> > hello, I tried this. but it still failed. how do you fix this error?
>
> 你把那个脚本下载到你本地安装目录下，然后 `load_dataset(csv_script_path, data_fiels)`

好的好的！解决了，感谢感谢！！！

>
>
> > hello, how did you solve this problems?
> > > > > Hi !
> > > > > The url works on my side.
> > > > > Is the url working in your navigator ?
> > > > > Are you connected to internet ? Does your network block access to `raw.githubusercontent.com` ?
> > > >
> > > >
> > > > I tried another server, it's working now. Thanks a lot.
> > > > And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?
> > >
> > >
> > > I download the scripts `https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py` and move it to the package dir `*/datasets/` solved the problem. Could you please put the file `datasets/datasets/csv/csv.py` to `datasets/src/datasets/`？
> > > Thanks :D
> >
> >
> > hello, I tried this. but it still failed. how do you fix this error?
>
> 你把那个脚本下载到你本地安装目录下，然后 `load_dataset(csv_script_path, data_fiels)`

我照着做了，然后报错。
ValueError: unable to parse C:/Software/Anaconda/envs/ptk_gpu2/Lib/site-packages/datasets\dataset_infos.json as a URL or as a local path

`---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-fd2106a3f053> in <module>
----> 1 dataset = load_dataset('C:/Software/Anaconda/envs/ptk_gpu2/Lib/site-packages/datasets/csv.py', data_files='./test.csv', delimiter=',', autogenerate_column_names=False)

C:\Software\Anaconda\envs\ptk_gpu2\lib\site-packages\datasets\load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, save_infos, script_version, **config_kwargs)
    588     # Download/copy dataset processing script
    589     module_path, hash = prepare_module(
--> 590         path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
    591     )
    592

C:\Software\Anaconda\envs\ptk_gpu2\lib\site-packages\datasets\load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, **download_kwargs)
    296         local_dataset_infos_path = cached_path(
    297             dataset_infos,
--> 298             download_config=download_config,
    299         )
    300     except (FileNotFoundError, ConnectionError):

C:\Software\Anaconda\envs\ptk_gpu2\lib\site-packages\datasets\utils\file_utils.py in cached_path(url_or_filename, download_config, **download_kwargs)
    316     else:
    317         # Something unknown
--> 318         raise ValueError("unable to parse {} as a URL or as a local path".format(url_or_filename))
    319
    320     if download_config.extract_compressed_file and output_path is not None:

ValueError: unable to parse C:/Software/Anaconda/envs/ptk_gpu2/Lib/site-packages/datasets\dataset_infos.json as a URL or as a local path

`

I also experienced this issue this morning. Looks like something specific to windows.
I'm working on a fix

>
>
> I opened a PR @wn1652400018

Thanks you!, It works very well.

## load_dataset for LOCAL CSV files report CONNECTION ERROR
- **Description:**
A local demo csv file:
```
import pandas as pd
import numpy as np
from datasets import load_dataset
import torch
import transformers

df = pd.DataFrame(np.arange(1200).reshape(300,4))
df.to_csv('test.csv', header=False, index=False)

print('datasets version: ', datasets.__version__)
print('pytorch version: ', torch.__version__)
print('transformers version: ', transformers.__version__)

# output:
datasets version:  1.1.2
pytorch version:  1.5.0
transformers version:  3.2.0
```

when I load data through `dataset`:
```
dataset = load_dataset('csv', data_files='./test.csv', delimiter=',', autogenerate_column_names=False)
```
Error infos:
```
ConnectionError                           Traceback (most recent call last)
<ipython-input-17-bbdadb9a0c78> in <module>
----> 1 dataset = load_dataset('csv', data_files='./test.csv', delimiter=',', autogenerate_column_names=False)

~/.conda/envs/py36/lib/python3.6/site-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, save_infos, script_version, **config_kwargs)
    588     # Download/copy dataset processing script
    589     module_path, hash = prepare_module(
--> 590         path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
    591     )
    592

~/.conda/envs/py36/lib/python3.6/site-packages/datasets/load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, **download_kwargs)
    266         file_path = hf_github_url(path=path, name=name, dataset=dataset, version=script_version)
    267         try:
--> 268             local_path = cached_path(file_path, download_config=download_config)
    269         except FileNotFoundError:
    270             if script_version is not None:

~/.conda/envs/py36/lib/python3.6/site-packages/datasets/utils/file_utils.py in cached_path(url_or_filename, download_config, **download_kwargs)
    306             user_agent=download_config.user_agent,
    307             local_files_only=download_config.local_files_only,
--> 308             use_etag=download_config.use_etag,
    309         )
    310     elif os.path.exists(url_or_filename):

~/.conda/envs/py36/lib/python3.6/site-packages/datasets/utils/file_utils.py in get_from_cache(url, cache_dir, force_download, proxies, etag_timeout, resume_download, user_agent, local_files_only, use_etag)
    473         elif response is not None and response.status_code == 404:
    474             raise FileNotFoundError("Couldn't find file at {}".format(url))
--> 475         raise ConnectionError("Couldn't reach {}".format(url))
    476
    477     # Try a second time

ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py
```

And I try to connect to the site with requests:
```
import requests

requests.head("https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py")
```

Similarly Error occurs:
```
---------------------------------------------------------------------------
ConnectionRefusedError                    Traceback (most recent call last)
~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connection.py in _new_conn(self)
    159             conn = connection.create_connection(
--> 160                 (self._dns_host, self.port), self.timeout, **extra_kw
    161             )

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/util/connection.py in create_connection(address, timeout, source_address, socket_options)
     83     if err is not None:
---> 84         raise err
     85

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/util/connection.py in create_connection(address, timeout, source_address, socket_options)
     73                 sock.bind(source_address)
---> 74             sock.connect(sa)
     75             return sock

ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

NewConnectionError                        Traceback (most recent call last)
~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    676                 headers=headers,
--> 677                 chunked=chunked,
    678             )

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    380         try:
--> 381             self._validate_conn(conn)
    382         except (SocketTimeout, BaseSSLError) as e:

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py in _validate_conn(self, conn)
    975         if not getattr(conn, "sock", None):  # AppEngine might not have  `.sock`
--> 976             conn.connect()
    977

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connection.py in connect(self)
    307         # Add certificate verification
--> 308         conn = self._new_conn()
    309         hostname = self.host

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connection.py in _new_conn(self)
    171             raise NewConnectionError(
--> 172                 self, "Failed to establish a new connection: %s" % e
    173             )

NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f3cceda5e48>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

MaxRetryError                             Traceback (most recent call last)
~/.conda/envs/py36/lib/python3.6/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    448                     retries=self.max_retries,
--> 449                     timeout=timeout
    450                 )

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    724             retries = retries.increment(
--> 725                 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
    726             )

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
    438         if new_retry.is_exhausted():
--> 439             raise MaxRetryError(_pool, url, error or ResponseError(cause))
    440

MaxRetryError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /huggingface/datasets/1.1.2/datasets/csv/csv.py (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f3cceda5e48>: Failed to establish a new connection: [Errno 111] Connection refused',))

During handling of the above exception, another exception occurred:

ConnectionError                           Traceback (most recent call last)
<ipython-input-20-18cc3eb4a049> in <module>
      1 import requests
      2
----> 3 requests.head("https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py")

~/.conda/envs/py36/lib/python3.6/site-packages/requests/api.py in head(url, **kwargs)
    102
    103     kwargs.setdefault('allow_redirects', False)
--> 104     return request('head', url, **kwargs)
    105
    106

~/.conda/envs/py36/lib/python3.6/site-packages/requests/api.py in request(method, url, **kwargs)
     59     # cases, and look like a memory leak in others.
     60     with sessions.Session() as session:
---> 61         return session.request(method=method, url=url, **kwargs)
     62
     63

~/.conda/envs/py36/lib/python3.6/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    528         }
    529         send_kwargs.update(settings)
--> 530         resp = self.send(prep, **send_kwargs)
    531
    532         return resp

~/.conda/envs/py36/lib/python3.6/site-packages/requests/sessions.py in send(self, request, **kwargs)
    641
    642         # Send the request
--> 643         r = adapter.send(request, **kwargs)
    644
    645         # Total elapsed time of the request (approximately)

~/.conda/envs/py36/lib/python3.6/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    514                 raise SSLError(e, request=request)
    515
--> 516             raise ConnectionError(e, request=request)
    517
    518         except ClosedPoolError as e:

ConnectionError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /huggingface/datasets/1.1.2/datasets/csv/csv.py (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f3cceda5e48>: Failed to establish a new connection: [Errno 111] Connection refused',))
```


Quail dataset urls are out of date

Hi ! Thanks for reporting.
We should fix the urls and use quail 1.3.
If you want to contribute feel free to fix the urls and open a PR :)

Done! PR [https://github.com/huggingface/datasets/pull/820](https://github.com/huggingface/datasets/pull/820)

Updated links and also regenerated the metadata and dummy data for v1.3 in order to pass verifications as described here: [https://huggingface.co/docs/datasets/share_dataset.html#adding-tests-and-metadata-to-the-dataset](https://huggingface.co/docs/datasets/share_dataset.html#adding-tests-and-metadata-to-the-dataset).

Closing since #820 is merged.
Thanks again for fixing the urls :)

Code

```
from datasets import load_dataset
quail = load_dataset('quail')
```

Error

```
FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/text-machine-lab/quail/master/quail_v1.2/xml/ordered/quail_1.2_train.xml
```

As per [quail v1.3 commit](https://github.com/text-machine-lab/quail/commit/506501cfa34d9ec6c042d31026ba6fea6bcec8ff) it looks like the location and suggested ordering has changed. In [https://github.com/huggingface/datasets/blob/master/datasets/quail/quail.py#L52-L58](https://github.com/huggingface/datasets/blob/master/datasets/quail/quail.py#L52-L58) the quail v1.2 datasets are being pointed to, which don't exist anymore.


On loading a metric from datasets, I get the following error

Hi ! We support only pyarrow > 0.17.1 so that we have access to the `PyExtensionType` object.
Could you update pyarrow and try again ?
```
pip install --upgrade pyarrow
```

`from datasets import load_metric`

`metric = load_metric('bleurt')`

Traceback:
210 class _ArrayXDExtensionType(pa.PyExtensionType):
    211
    212     ndims: int = None

AttributeError: module 'pyarrow' has no attribute 'PyExtensionType'

Any help will be appreciated. Thank you.


Empty output/answer in TriviaQA test set (both in 'kilt_tasks' and 'trivia_qa')

Yes: TriviaQA has a private test set for the leaderboard [here](https://competitions.codalab.org/competitions/17208)

For the KILT training and validation portions, you need to link the examples from the TriviaQA dataset as detailed here:
https://github.com/huggingface/datasets/blob/master/datasets/kilt_tasks/README.md

Oh ok, I guess I read the paper too fast 😅, thank you for your answer!

# The issue

It's all in the title, it appears to be fine on the train and validation sets.

Is there some kind of mapping to do like for the questions (see https://github.com/huggingface/datasets/blob/master/datasets/kilt_tasks/README.md) ?

# How to reproduce
```py
from datasets import load_dataset
kilt_tasks = load_dataset("kilt_tasks")
trivia_qa = load_dataset('trivia_qa', 'unfiltered.nocontext')
# both in "kilt_tasks"
In [18]: any([output['answer'] for output in kilt_tasks['test_triviaqa']['output']])
Out[18]: False
# and "trivia_qa"
In [13]: all([answer['value'] == '<unk>' for answer in trivia_qa['test']['answer']])
Out[13]: True
# appears to be fine on the train and validation sets.
In [14]: all([answer['value'] == '<unk>' for answer in trivia_qa['train']['answer']])
Out[14]: False

In [15]: all([answer['value'] == '<unk>' for answer in trivia_qa['validation']['answer']])
Out[15]: False

In [16]: any([output['answer'] for output in kilt_tasks['train_triviaqa']['output']])
Out[16]: True

In [17]: any([output['answer'] for output in kilt_tasks['validation_triviaqa']['output']])
Out[17]: True

```


fix: typos in tutorial to map KILT and TriviaQA


Add XGlue

Really cool to add XGlue, this will be a nice addition !

Splits shouldn't depend on the language. There must be configurations for each language, as we're doing for xnli, xtreme, etc.
For example for XGlue we'll have these configurations: NER.de, NER.en etc.

Dataset is ready to merge. An important feature of this dataset is that for each config the train data is in English, while dev and test data are in multiple languages. Therefore, @lhoestq and I decided offline that we will give the dataset the following API, *e.g.* for

```python
load_dataset("xglue", "ner")  # would give the splits 'train', 'validation.en', 'test.en', 'validation.es', 'test.es', ...
```
=> therefore one can load a single language test via

```python
load_dataset("xglue", "ner", split="test.es")
```


How to join two datasets?

Hi ! Currently the only way to add new fields to a dataset is by using `.map` and picking items from the other dataset

Closing this one. Feel free to re-open if you have other questions about this issue.

Also linking another discussion about joining datasets: #853

Hi,

I'm wondering if it's possible to join two (preprocessed) datasets with the same number of rows but different labels?

I'm currently trying to create paired sentences for BERT from `wikipedia/'20200501.en`, and I couldn't figure out a way to create a paired sentence using `.map()` where the second sentence is **not** the next sentence (i.e., from a different article) of the first sentence.

Thanks!


Update loading_metrics.rst

Minor bug


switch amazon reviews class label order

Switches the label order to be more intuitive for amazon reviews, #791.