Kokolipa/github-issues huggingface dataset:
Issue: How to choose proper download_mode in function load_dataset?
State: closed

Hi, I am a beginner to datasets and I try to use datasets to load my csv file.
my csv file looks like this

``` 
text,label
"Effective but too-tepid biopic",3
"If you sometimes like to go to the movies to have fun , Wasabi is a good place to start .",4
"Emerges as something rare , an issue movie that 's so honest and keenly observed that it does n't feel like one .",5
```

First I try to use this command to load my csv file .  

``` python
dataset=load_dataset('csv', data_files=['sst_test.csv'])
```

It seems good, but when i try to overwrite the convert_options to convert  'label' columns from int64 to float32 like this.

``` python
import pyarrow as pa
from pyarrow import csv
read_options = csv.ReadOptions(block_size=1024*1024)
parse_options = csv.ParseOptions()
convert_options = csv.ConvertOptions(column_types={'text': pa.string(), 'label': pa.float32()})
dataset = load_dataset('csv', data_files=['sst_test.csv'], read_options=read_options,
                       parse_options=parse_options, convert_options=convert_options)
```

It keeps the same:

```shell
Dataset(features: {'text': Value(dtype='string', id=None), 'label': Value(dtype='int64', id=None)}, num_rows: 2210)
```

I think this issue is caused by the parameter "download_mode" Default to REUSE_DATASET_IF_EXISTS because after I delete the cache_dir, it seems right.

Is it a bug? How to choose proper download_mode to avoid this issue?


`download_mode=datasets.GenerateMode.FORCE_REDOWNLOAD` should work.
This makes me think we we should rename this to DownloadMode.FORCE_REDOWNLOAD. Currently that's confusing

Can we just use `features=...` in `load_dataset` for this @lhoestq?

Indeed you should use `features` in this case. 
```python
features = Features({'text': Value('string'), 'label': Value('float32')})
dataset = load_dataset('csv', data_files=['sst_test.csv'], features=features)
```
Note that because of an issue with the caching when you change the features (see #750 ) you still need to specify the `FORCE_REDOWNLOAD ` flag. I'm working on a fix for this one

https://github.com/huggingface/datasets/issues/769#issuecomment-717837832
> This makes me think we we should rename this to DownloadMode.FORCE_REDOWNLOAD. Currently that's confusing

@lhoestq do you still think we should rename it?


It's no big deal, but since it can be confusing to users I think it's worth renaming it, and deprecate `GenerateMode` until `datasets` 2.0 at least. IMO it's confusing to have `download_mode=GenerateMode.something`
Issue: Add a `lazy_map` method to `Dataset` and `DatasetDict`
State: open
Labels: name: enhancement

The library is great, but it would be even more awesome with a `lazy_map` method implemented on `Dataset` and `DatasetDict`. This would apply a function on a give item but when the item is requested. Two use cases:

1. load image on the fly
2. apply a random function and get different outputs at each epoch (like data augmentation or randomly masking a part of a sentence for BERT-like objectives).

This is cool! I think some aspects to think about and decide in terms of API are:
- do we allow several methods (chained i guess)
- how do we inspect the currently set method(s)
- how do we control/reset them
Issue: Add option for named splits when using ds.train_test_split
State: open
Labels: name: enhancement

### Feature Request 🚀 

Can we add a way to name your splits when using the `.train_test_split` function?

In almost every use case I've come across, I have a `train` and a `test` split in my `DatasetDict`, and I want to create a `validation` split. Therefore, its kinda useless to get a `test` split back from `train_test_split`, as it'll just overwrite my real `test` split that I intended to keep.

### Workaround

this is my hack for dealin with this, for now :slightly_smiling_face:

```python
from datasets import load_dataset
​
​
ds = load_dataset('imdb')
ds['train'], ds['validation'] = ds['train'].train_test_split(.1).values()
```


Yes definitely we should give more flexibility to control the name of the splits outputted by `train_test_split`.

Related is the very interesting feedback from @bramvanroy on how we should improve this method: https://discuss.huggingface.co/t/how-to-split-main-dataset-into-train-dev-test-as-datasetdict/1090/5

And in particular that it should advantageously be able to split in 3 splits as well instead of just 2 like we copied from sklearn.
Issue: [GEM] add DART data-to-text generation dataset
State: closed
Labels: name: dataset request

## Adding a Dataset
- **Name:** DART
- **Description:** DART consists of 82,191 examples across different domains with each input being a semantic RDF triple set derived from data records in tables and the tree ontology of the schema, annotated with sentence descriptions that cover all facts in the triple set.
- **Paper:** https://arxiv.org/abs/2007.02871v1
- **Data:** https://github.com/Yale-LILY/dart
- **Motivation:** the dataset will likely be included in the GEM benchmark

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).


Is this a duplicate of #924 ?

Yup, closing! Haven't been keeping track of the solved issues during the sprint.
Issue: [GEM] Add DART data-to-text generation dataset
State: closed
Labels: name: dataset request

## Adding a Dataset
- **Name:** DART
- **Description:** DART consists of 82,191 examples across different domains with each input being a semantic RDF triple set derived from data records in tables and the tree ontology of the schema, annotated with sentence descriptions that cover all facts in the triple set.
- **Paper:** https://arxiv.org/abs/2007.02871v1
- **Data:** https://github.com/Yale-LILY/dart
- **Motivation:** It will likely be included in the GEM generation evaluation benchmark

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).
Issue: Downloaded datasets are not usable offline
State: closed

I've been trying to use the IMDB dataset offline, but after downloading it and turning off the internet it still raises  an error from the ```requests``` library trying to reach for the online dataset.
Is this the intended behavior ?
(Sorry, I wrote the the first version of this issue while still on nlp 0.3.0).

Yes currently you need an internet connection because the lib tries to check for the etag of the dataset script online to see if you don't have it locally already.

If we add a way to store the etag/hash locally after the first download, it would allow users to first download the dataset with an internet connection, and still have it working without an internet connection.

I'll let you know when we add this feature.

Already fixed by:
- #1726
Issue: Process 0 very slow when using num_procs with map to tokenizer
State: closed

<img width="721" alt="image" src="https://user-images.githubusercontent.com/17930170/97066109-776d0d00-15ed-11eb-8bba-bb4d2e0fcc33.png">
The code I am using is
```

        dataset = load_dataset("text", data_files=[file_path], split='train')
        dataset = dataset.map(lambda ex: tokenizer(ex["text"], add_special_tokens=True,
                                                truncation=True, max_length=args.block_size), num_proc=8)
        dataset.set_format(type='torch', columns=['input_ids'])
        dataset.save_to_disk(file_path+'.arrow')
```


Hi ! Thanks for reporting.
Is the distribution of text length of your data evenly distributed across your dataset ? I mean, could it be because the examples in the first part of your dataset are slower to process ?
Also could how many CPUs can you use for multiprocessing ?
```python
import multiprocessing
print(multiprocessing.cpu_count())
```
Which tokenizer are you using ?

Using pre trained HF tokenizer. The result is the same with tokenizer multiprocessing off and on.
I have (absolutely) no idea about the distribution, but since this issue occurs on all of my datasets(regardless of files), I don't think distribution is the problems.

I can use up to 16 cores.

Ok weird, I don't manage to reproduce this issue on my side.
Does it happen even with `num_proc=2` for example ?
Also could you provide more details about your OS and the versions of tokenizers/datasets/multiprocess that you're using ?

Yes, I can confirm it also happens with ```num_proc=2```.
```
tokenizers               0.9.2
datasets                 1.1.2
multiprocess             0.70.10
```
```
Linux nipa2020-0629 4.4.0-178-generic #208-Ubuntu SMP Sun Apr 5 23:45:10 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
```

I can't reproduce on my side unfortunately with the same versions.

Do you have issues when doing multiprocessing with python ?
```python
from tqdm.auto import tqdm
from multiprocess import Pool, RLock

def process_data(shard):
    # implement

num_proc = 8
shards = [] # implement, this must be a list of size num_proc

with Pool(num_proc, initargs=(RLock(),), initializer=tqdm.set_lock) as pool:
    results = [pool.apply_async(process_data, shard=shard) for shard in shards]
    transformed_shards = [r.get() for r in results]
```

Nah, I'll just wait a few hours. Thank you for helping, though.
Issue: CUDA out of memory
State: closed

In your dataset ,cuda run out of memory as long as the trainer begins:
however, without changing any other element/parameter,just switch dataset to `LineByLineTextDataset`,everything becames OK.


Could you provide more details ? What's the code you ran ?

```python
tokenizer = FunnelTokenizer.from_pretrained('funnel-transformer/small')

def tokenize(batch):
    return tokenizer(batch['text'], padding='max_length', truncation=True,max_length=512)

dataset = load_dataset("bookcorpus",split='train[:1000]').shuffle()
dataset = dataset.map(tokenize, batched=True, batch_size=512)

# dataset = LineByLineTextDataset(
#     tokenizer=tokenizer,
#     file_path="./wiki1000.txt",
#     block_size=128
# )

data_collator = DataCollatorForLanguageModeling(
    tokenizer=tokenizer, mlm=True, mlm_probability=0.15
)

config=FunnelConfig(
    return_dict=True
)

model= FunnelForMaskedLM(config=config)

training_args = TrainingArguments(
    output_dir="./checkpoints",
    overwrite_output_dir=True,
    do_train=True,
    num_train_epochs=1,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    save_steps=10000,
    logging_dir='./ptlogs'
)

trainer = Trainer(
    model=model,
    args=training_args,
    data_collator=data_collator,
    train_dataset=dataset,
)
trainer.train()
```

`RuntimeError: CUDA out of memory. Tried to allocate 954.00 MiB (GPU 0; 15.90 GiB total capacity; 14.35 GiB already allocated; 753.75 MiB free; 14.39 GiB reserved in total by PyTorch)
Exception raised from malloc at /pytorch/c10/cuda/CUDACachingAllocator.cpp:272 (most recent call first):`

part of error output

from funnel model to bert model : error still happened

from your dataset to LineByLineTextDataset : error disapeared

notice i just loaded 1000 rows of data

the error happens when executing loss.backward()

Since you're using a data collator you don't need to tokenizer the dataset using `map`. Could you try not to use `map` and only the data collator instead ? The data collator is supposed to pad to the longest sequence in each batch afaik, instead of padding to 512.

Also cc @sgugger 

Closing this one.
Feel free to re-open if you have other questions about this issue
Issue: Clicking on a metric in the search page points to datasets page giving "Missing dataset" warning
State: closed

Hi! Sorry if this isn't the right place to talk about the website, I just didn't exactly where to write this.

Searching a metric in https://huggingface.co/metrics gives the right results but clicking on a metric (E.g ROUGE) points to https://huggingface.co/datasets/rouge. Clicking on a metric without searching points to the right page.

Thanks for all the great work!

Thanks for the report, can reproduce. Will fix

Fixed now @ogabrielluiz
Issue: Error loading ms_marco v2.1 using load_dataset()
State: closed

Code:
`dataset = load_dataset('ms_marco', 'v2.1')`

Error:
```
`---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
<ipython-input-16-34378c057212> in <module>()
      9 
     10 # Downloading and loading a dataset
---> 11 dataset = load_dataset('ms_marco', 'v2.1')

10 frames
/usr/lib/python3.6/json/decoder.py in raw_decode(self, s, idx)
    353         """
    354         try:
--> 355             obj, end = self.scan_once(s, idx)
    356         except StopIteration as err:
    357             raise JSONDecodeError("Expecting value", s, err.value) from None

JSONDecodeError: Unterminated string starting at: line 1 column 388988661 (char 388988660)
`
```

There was a similar issue in #294 
Clearing the cache and download again the dataset did the job. Could you try to clear your cache and download the dataset again ?

I was able to load the dataset successfully, I'm pretty sure it's just a cache issue that you have.
Let me know if clearing your cache fixes the problem

Yes, it indeed was a cache issue!
Thanks for reaching out!!
Issue: load_dataset doesn't include `features` in its hash
State: closed

It looks like the function `load_dataset` does not include what's passed in the `features` argument when creating a hash for a given  dataset. As a result, if a user includes new features from an already downloaded dataset, those are ignored.

Example: some models on the hub have a different ordering for the labels than what `datasets` uses for MNLI so I'd like to do something along the lines of:
```
dataset = load_dataset("glue", "mnli")
features = dataset["train"].features
features["label"] = ClassLabel(names = ['entailment', 'contradiction', 'neutral'])  # new label order
dataset = load_dataset("glue", "mnli", features=features)
```
Issue: Dataset Explorer Doesn't Work for squad_es and squad_it
State: closed
Labels: name: nlp-viewer

https://huggingface.co/nlp/viewer/?dataset=squad_es
https://huggingface.co/nlp/viewer/?dataset=squad_it

Both pages show "OSError: [Errno 28] No space left on device".

Oups wrong click.
This one is for you @srush
Issue: Trec Dataset Connection Error
State: closed

**Datasets Version:**
1.1.2

**Python Version:**
3.6/3.7


**Code:**
```python
from datasets import load_dataset
load_dataset("trec")
```

**Expected behavior:**
Download Trec dataset and load Dataset object

**Current Behavior:**
Get a connection error saying it couldn't reach http://cogcomp.org/Data/QA/QC/train_5500.label (but the link doesn't seem broken)

<details>
  <summary>Error Logs</summary>
 

Using custom data configuration default
Downloading and preparing dataset trec/default (download: 350.79 KiB, generated: 403.39 KiB, post-processed: Unknown size, total: 754.18 KiB) to /root/.cache/huggingface/datasets/trec/default/1.1.0/ca4248481ad244f235f4cf277186cad2ee8769f975119a2bbfc41b8932b88bd7...
---------------------------------------------------------------------------
ConnectionError                           Traceback (most recent call last)
<ipython-input-8-66bf1242096e> in <module>()
----> 1 load_dataset("trec")

10 frames
/usr/local/lib/python3.6/dist-packages/datasets/utils/file_utils.py in get_from_cache(url, cache_dir, force_download, proxies, etag_timeout, resume_download, user_agent, local_files_only, use_etag)
    473         elif response is not None and response.status_code == 404:
    474             raise FileNotFoundError("Couldn't find file at {}".format(url))
--> 475         raise ConnectionError("Couldn't reach {}".format(url))
    476 
    477     # Try a second time

ConnectionError: Couldn't reach http://cogcomp.org/Data/QA/QC/train_5500.label

</details>

Thanks for reporting.
That's because the download url has changed. The old url now redirects to the new one but we don't support redirection for downloads.

I'm opening a PR to update the url
Issue: Throw error when an unexpected key is used in data_files
State: closed

I have found that only "train", "validation" and "test" are valid keys in the `data_files` argument. When you use any other ones, those attached files are silently ignored - leading to unexpected behaviour for the users.

So the following, unintuitively, returns only one key (namely `train`).

```python
datasets = load_dataset("text", data_files={"train": train_f, "valid": valid_f})
print(datasets.keys())
# dict_keys(['train'])
```

whereas using `validation` instead, does return the expected result:

```python
datasets = load_dataset("text", data_files={"train": train_f, "validation": valid_f})
print(datasets.keys())
# dict_keys(['train', 'validation'])
```

I would like to see more freedom in which keys one can use, but if that is not possible at least an error should be thrown when using an unexpected key.

Thanks for reporting !
We'll add support for other keys
Issue: Better error message when one forgets to call `add_batch` before `compute`
State: closed

When using metrics, if for some reason a user forgets to call `add_batch` to a metric before `compute` (with no arguments), the error message is a bit cryptic and could probably be made clearer.

## Reproducer

```python
import datasets
import torch
from datasets import Metric

class GatherMetric(Metric):
    def _info(self):
        return datasets.MetricInfo(
            description="description",
            citation="citation",
            inputs_description="kwargs",
            features=datasets.Features({
                'predictions': datasets.Value('int64'),
                'references': datasets.Value('int64'),
            }),
            codebase_urls=[],
            reference_urls=[],
            format='numpy'
        )

    def _compute(self, predictions, references):
        return {"predictions": predictions, "labels": references}

metric = GatherMetric(cache_dir="test-metric")
inputs = torch.randint(0, 2, (1024,))
targets = torch.randint(0, 2, (1024,))

batch_size = 8
for i in range(0, 1024, batch_size):
     pass # User forgets to call `add_batch`
result = metric.compute()
```

## Stack trace:

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-267729d187fa> in <module>
      3      pass
      4     # metric.add_batch(predictions=inputs[i:i+batch_size], references=targets[i:i+batch_size])
----> 5 result = metric.compute()

~/git/datasets/src/datasets/metric.py in compute(self, *args, **kwargs)
    380         if predictions is not None:
    381             self.add_batch(predictions=predictions, references=references)
--> 382         self._finalize()
    383 
    384         self.cache_file_name = None

~/git/datasets/src/datasets/metric.py in _finalize(self)
    343         elif self.process_id == 0:
    344             # Let's acquire a lock on each node files to be sure they are finished writing
--> 345             file_paths, filelocks = self._get_all_cache_files()
    346 
    347             # Read the predictions and references

~/git/datasets/src/datasets/metric.py in _get_all_cache_files(self)
    280         filelocks = []
    281         for process_id, file_path in enumerate(file_paths):
--> 282             filelock = FileLock(file_path + ".lock")
    283             try:
    284                 filelock.acquire(timeout=self.timeout)

TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
```
Issue: Parallel downloads progress bar flickers
State: open

When there are parallel downloads using the download manager, the tqdm progress bar flickers since all the progress bars are on the same line.

To fix that we could simply specify `position=i` for i=0 to n the number of files to download when instantiating the tqdm progress bar. 

Another way would be to have one "master" progress bar that tracks the number of finished downloads, and then one progress bar per process that show the current downloads.
Issue: need to redirect /nlp to /datasets and remove outdated info
State: closed

It looks like the website still has all the `nlp` data, e.g.: https://huggingface.co/nlp/viewer/?dataset=wikihow&config=all

should probably redirect to: https://huggingface.co/datasets/wikihow

also for some reason the new information is slightly borked. If you look at the old one it was nicely formatted and had the links marked up, the new one is just a jumble of text in one chunk and no markup for links (i.e. not clickable).

Should be fixed now: 

![image](https://user-images.githubusercontent.com/35882/95917301-040b0600-0d78-11eb-9655-c4ac0e788089.png)

Not sure I understand what you mean by the second part?


Thank you!

> Not sure I understand what you mean by the second part?

Compare the 2:
* https://huggingface.co/datasets/wikihow
* https://huggingface.co/nlp/viewer/?dataset=wikihow&config=all
Can you see the difference? 2nd has formatting, 1st doesn't.


For context, those are two different pages (not an old vs new one), one is from the dataset viewer (you can browse data inside the datasets) while the other is just a basic reference page displayed some metadata about the dataset.

For the second one, we'll move to markdown parsing soon, so it'll be formatted better.

I understand. I was just flagging the lack of markup issue.
Issue: Adding pseudo-labels to datasets
State: closed

I recently [uploaded pseudo-labels](https://github.com/huggingface/transformers/blob/master/examples/seq2seq/precomputed_pseudo_labels.md) for CNN/DM, XSUM and WMT16-en-ro to s3, and thom mentioned I should add them to this repo.
Since pseudo-labels are just a large model's generations on an existing dataset, what is the right way to structure this contribution.
I read https://huggingface.co/docs/datasets/add_dataset.html, but it doesn't really cover this type of contribution.

I could, for example, make a new directory, `xsum_bart_pseudolabels` for each set of pseudolabels or add some sort of parametrization to `xsum.py`: https://github.com/huggingface/datasets/blob/5f4c6e830f603830117877b8990a0e65a2386aa6/datasets/xsum/xsum.py

What do you think @lhoestq ?




Nice ! :)
It's indeed the first time we have such contributions so we'll have to figure out the appropriate way to integrate them.
Could you add details on what they could be used for ?


They can be used as training data for a smaller model.

Sounds just like a regular dataset to me then, no?

A new configuration for those datasets should do the job then.
Note that until now datasets like xsum only had one configuration. It means that users didn't have to specify the configuration name when loading the dataset. If we add new configs, users that update the lib will have to update their code to specify the default/standard configuration name (not the one with pseudo labels).

Could also be a `user-namespace` dataset maybe?

Oh yes why not. I'm more in favor of this actually since pseudo labels are things that users (not dataset authors in general) can compute by themselves and share with the community

![image](https://user-images.githubusercontent.com/6045025/96045248-b528a380-0e3f-11eb-9124-bd55afa031bb.png)

I assume I should (for example) rename the xsum dir, change the URL, and put the modified dir somewhere in S3?

You can use the `datasets-cli` to upload the folder with your version of xsum with the pseudo labels.

```
datasets-cli upload_dataset path/to/xsum
```
Issue: Error in the notebooks/Overview.ipynb notebook
State: closed

Hi,

I got the following error in **cell number 3** while exploring the **Overview.ipynb** notebook in google colab. I used the [link ](https://colab.research.google.com/github/huggingface/datasets/blob/master/notebooks/Overview.ipynb) provided in the main README file to open it in colab. 

```python
# You can access various attributes of the datasets before downloading them
squad_dataset = list_datasets()[datasets.index('squad')]

pprint(squad_dataset.__dict__)  # It's a simple python dataclass
```

Error message
```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-5-8dc805c4949c> in <module>()
      2 squad_dataset = list_datasets()[datasets.index('squad')]
      3 
 ----> 4 pprint(squad_dataset.__dict__)  # It's a simple python dataclass
    
AttributeError: 'str' object has no attribute '__dict__'
```

The object `squad_dataset` is a `str` not a `dataclass` .

Do this:
``` python
squad_dataset = list_datasets(with_details=True)[datasets.index('squad')]
pprint(squad_dataset.__dict__)  # It's a simple python dataclass
```

Thanks! This worked. I have created a PR to fix this in the notebook.
Issue: How to use similarity settings  other then "BM25" in Elasticsearch index ?
State: closed

**QUESTION : How should we use other similarity algorithms supported by Elasticsearch other than "BM25"  ?**
**ES Reference**
https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-similarity.html
**HF doc reference:**
https://huggingface.co/docs/datasets/faiss_and_ea.html

**context :**
========

I used the latest Elasticsearch server  version 7.9.2
When I set DFR  which is one of the other similarity algorithms supported by elasticsearch  in the mapping, I get an error

For example DFR that I had tried in the first instance in mappings as below.,
`"mappings": {"properties": {"text": {"type": "text", "analyzer": "standard", "similarity": "DFR"}}},`

I get the following error 
RequestError: RequestError(400, 'mapper_parsing_exception', 'Unknown Similarity type [DFR] for field [text]')

The other thing as another option I had tried was to declare "similarity": "my_similarity" within settings and then assigning "my_similarity" inside the mappings as below 

`es_config = {
        "settings": {
            "number_of_shards": 1,
             **"similarity":  "my_similarity"**: {
          "type": "DFR",
          "basic_model": "g",
          "after_effect": "l",
          "normalization": "h2",
          "normalization.h2.c": "3.0"
        } ,
            "analysis": {"analyzer": {"stop_standard": {"type": "standard", " stopwords": "_english_"}}},
            
        },
        "mappings": {"properties": {"text": {"type": "text", "analyzer": "standard", "similarity": "my_similarity"}}},
    }`

For this , I got the following error
RequestError: RequestError(400, 'illegal_argument_exception', 'unknown setting [index.similarity] please check that any required plugins are installed, or check the breaking changes documentation for removed settings')



Datasets does not use elasticsearch API to define custom similarity. If you want to use a custom similarity, the best would be to run a curl request directly to your elasticsearch instance (see sample hereafter, directly from ES documentation), then you should be able to use `my_similarity` in your configuration passed to datasets

```
curl -X PUT "localhost:9200/index?pretty" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "index": {
      "similarity": {
        "my_similarity": {
          "type": "DFR",
          "basic_model": "g",
          "after_effect": "l",
          "normalization": "h2",
          "normalization.h2.c": "3.0"
        }
      }
    }
  }
}
'

```
Issue: Requirements should specify pyarrow<1
State: closed

I was looking at the docs on [Perplexity](https://huggingface.co/transformers/perplexity.html) via GPT2. When you load datasets and try to load Wikitext, you get the error,

```
module 'pyarrow' has no attribute 'PyExtensionType'
```
I traced it back to datasets having installed PyArrow 1.0.1 but there's not pinning in the setup file. 

https://github.com/huggingface/datasets/blob/e86a2a8f869b91654e782c9133d810bb82783200/setup.py#L68

Downgrading by installing `pip install "pyarrow<1"` resolved the issue.

Hello @mathcass I would want to work on this issue. May I do the same? 

@punitaojha, certainly. Feel free to work on this. Let me know if you need any help or clarity.

Hello @mathcass 
1. I did fork the repository and clone the same on my local system. 

2. Then learnt about how we can publish our package on pypi.org. Also, found some instructions on same in setup.py documentation.

3. Then I Perplexity document link that you shared above. I created a colab link from there keep both tensorflow and pytorch means a mixed option and tried to run it in colab but I encountered no errors at a point where you mentioned. Can you help me to figure out the issue. 

4.Here is the link of the colab file with my saved responses. 
https://colab.research.google.com/drive/1hfYz8Ira39FnREbxgwa_goZWpOojp2NH?usp=sharing

Also, please share some links which made you conclude that pyarrow < 1 would help. 

Access granted for the colab link. 

Thanks for looking at this @punitaojha and thanks for sharing the notebook. 

I just tried to reproduce this on my own (based on the environment where I had this issue) and I can't reproduce it somehow. If I run into this again, I'll include some steps to reproduce it. I'll close this as invalid. 

Thanks again. 

I am sorry for hijacking this closed issue, but I believe I was able to reproduce this very issue. Strangely enough, it also turned out that running `pip install "pyarrow<1" --upgrade` did indeed fix the issue (PyArrow was installed in version `0.14.1` in my case).

Please see the Colab below:

https://colab.research.google.com/drive/15QQS3xWjlKW2aK0J74eEcRFuhXUddUST

Thanks!
Issue: Add UI filter to filter datasets based on task
State: closed
Labels: name: enhancement

This is great work, so huge shoutout to contributors and huggingface.

The [/nlp/viewer](https://huggingface.co/nlp/viewer/) is great and the [/datasets](https://huggingface.co/datasets) page is great. I was wondering if in both or either places we can have a filter that selects if a dataset is good for the following tasks (non exhaustive list)

- Classification
	- Multi label
	- Multi class
- Q&A
- Summarization
- Translation

I believe this feature might have some value, for folks trying to find datasets for a particular task, and then testing their model capabilities.

Thank you :) 

Already supported.
Issue: XNLI dataset: NonMatchingChecksumError
State: closed

Hi,
I tried to download "xnli" dataset in colab using 
`xnli = load_dataset(path='xnli')`
but got 'NonMatchingChecksumError' error

`NonMatchingChecksumError                  Traceback (most recent call last)
<ipython-input-27-a87bedc82eeb> in <module>()
----> 1 xnli = load_dataset(path='xnli')

3 frames
/usr/local/lib/python3.6/dist-packages/datasets/utils/info_utils.py in verify_checksums(expected_checksums, recorded_checksums, verification_name)
     37     if len(bad_urls) > 0:
     38         error_msg = "Checksums didn't match" + for_verification_name + ":\n"
---> 39         raise NonMatchingChecksumError(error_msg + str(bad_urls))
     40     logger.info("All the checksums matched successfully" + for_verification_name)
     41 

NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://www.nyu.edu/projects/bowman/xnli/XNLI-1.0.zip']`

The same code worked well several days ago in colab but stopped working now. Thanks!

Thanks for reporting.
The data file must have been updated by the host.
I'll update the checksum with the new one.

Well actually it looks like the link isn't working anymore :(

The new link is https://cims.nyu.edu/~sbowman/xnli/XNLI-1.0.zip
I'll update the dataset script

I'll do a release in the next few days to make the fix available for everyone.
In the meantime you can load `xnli` with
```
xnli = load_dataset('xnli', script_version="master")
```
This will use the latest version of the xnli script (available on master branch), instead of the old one.

That's awesome! Thanks a lot!
Issue: Dataset browser url is still https://huggingface.co/nlp/viewer/
State: closed

Might be worth updating to https://huggingface.co/datasets/viewer/

Yes! might do it with @srush one of these days. Hopefully it won't break too many links (we can always redirect from old url to new)

This was fixed but forgot to close the issue. cc @lhoestq @yjernite 

Thanks @jarednielsen!
Issue: The download instructions for c4 datasets are not contained in the error message
State: closed

The manual download instructions are not clear 
```The dataset c4 with config en requires manual data. 
 Please follow the manual download instructions: <bound method C4.manual_download_instructions of <datasets_modules.datasets.c4.830b0c218bd41fed439812c8dd19dbd4767d2a3faa385eb695cf8666c982b1b3.c4.C4 object at 0x7ff8c5969760>>. 
 Manual data can be loaded with `datasets.load_dataset(c4, data_dir='<path/to/manual/data>')
```

Either `@property`  could be added to C4.manual_download_instrcutions (or make it a real property), or the manual_download_instructions function needs to be called I think.

Let me know if you want a PR for this, but I'm not sure which possible fix is the correct one.

Good catch !
Indeed the `@property` is missing.

Feel free to open a PR :)

Also not that C4 is a dataset that needs an Apache Beam runtime to be generated.
For example Dataflow, Spark, Flink etc.

Usually we generate the dataset on our side once and for all, but we haven't done it for C4 yet.
More info about beam datasets [here](https://huggingface.co/docs/datasets/beam_dataset.html)

Let me know if you have any questions
Issue: train_test_split returns empty dataset item
State: closed

I try to split my dataset by `train_test_split`, but after that the item in `train` and `test` `Dataset` is empty.
The codes:
```
yelp_data = datasets.load_from_disk('/home/ssd4/huanglianzhe/test_yelp')
    print(yelp_data[0])
    yelp_data = yelp_data.train_test_split(test_size=0.1)
    print(yelp_data)
    print(yelp_data['test'])
    print(yelp_data['test'][0])
```
The outputs:
```
{'stars': 2.0, 'text': 'xxxx'}
Loading cached split indices for dataset at /home/ssd4/huanglianzhe/test_yelp/cache-f9b22d8b9d5a7346.arrow and /home/ssd4/huanglianzhe/test_yelp/cache-4aa26fa4005059d1.arrow
DatasetDict({'train': Dataset(features: {'stars': Value(dtype='float64', id=None), 'text': Value(dtype='string', id=None)}, num_rows: 7219009), 'test': Dataset(features: {'stars': Value(dtype='float64', id=None), 'text': Value(dtype='string', id=None)}, num_rows: 802113)})
Dataset(features: {'stars': Value(dtype='float64', id=None), 'text': Value(dtype='string', id=None)}, num_rows: 802113)
{}    # yelp_data['test'][0] is empty
```

The problem still exists after removing the cache files.

Can you reproduce this example in a Colab so we can investigate? (or give more information on your software/hardware config)

Thanks for reporting.
I just found the issue, I'm creating a PR

We'll do a release pretty soon to include the fix :)
In the meantime you can install the lib from source if you want to
Issue: load_dataset() won't download in Windows
State: closed

I don't know if this is just me or Windows. Maybe other Windows users can chime in if they don't have this problem. I've been trying to get some of the tutorials working on Windows, but when I use the load_dataset() function, it just stalls and the script keeps running indefinitely without downloading anything. I've waited upwards of 18 hours to download the 'multi-news' dataset (which isn't very big), and still nothing. I've tried running it through different IDE's and the command line, but it had the same behavior. I've also tried it with all virus and malware protection turned off. I've made sure python and all IDE's are exceptions to the firewall and all the requisite permissions are enabled.

Additionally, I checked to see if other packages could download content such as an nltk corpus, and they could. I've also run the same script using Ubuntu and it downloaded fine (and quickly). When I copied the downloaded datasets from my Ubuntu drive to my Windows .cache folder it worked fine by reusing the already-downloaded dataset, but it's cumbersome to do that for every dataset I want to try in my Windows environment.

Could this be a bug, or is there something I'm doing wrong or not thinking of?

Thanks.

I have the same issue. Tried to download a few of them and not a single one is downloaded successfully.

This is the output:
```
>>> dataset = load_dataset('blended_skill_talk', split='train')
Using custom data configuration default               <-- This step never ends
```

This was fixed in #644 
I'll do a new release soon :)

In the meantime you can run it by installing from source

Closing since version 1.1.0 got released with Windows support :) 
Let me know if it works for you now
Issue: blog_authorship_corpus crashed
State: closed
Labels: name: nlp-viewer

This is just to report that When I pick blog_authorship_corpus in 
https://huggingface.co/nlp/viewer/?dataset=blog_authorship_corpus
I get this:
![image](https://user-images.githubusercontent.com/7553188/94349542-4364f300-0013-11eb-897d-b25660a449f0.png)



Thanks for reporting !
We'll free some memory
Issue: [BUG] No such file or directory
State: closed

This happens when both
1. Huggingface datasets cache dir does not exist
2. Try to load a local dataset script

builder.py throws an error when trying to create a filelock in a directory (cache/datasets) that does not exist
https://github.com/huggingface/datasets/blob/master/src/datasets/builder.py#L177

Tested on v1.0.2

@lhoestq
Issue: How to skip a example when running dataset.map
State: closed

in processing func, I process examples and detect some invalid examples, which I did not want it to be added into train dataset. However I did not find how to skip this recognized invalid example when doing dataset.map. 

Hi @xixiaoyao,
Depending on what you want to do you can:
- use a first step of `filter` to filter out the invalid examples: https://huggingface.co/docs/datasets/processing.html#filtering-rows-select-and-filter
- or directly detect the invalid examples inside the callable used with `map` and return them unchanged or even remove them at the same time if you are using `map` in batched mode. Here is an example where we use `map` in batched mode to add new rows on the fly but you can also use it to remove examples on the fly (that's what `filter` actually do under-the-hood): https://huggingface.co/docs/datasets/processing.html#augmenting-the-dataset

Closing this one.
Feel free to re-open if you have other questions

Letting finders-of-this-thread know that the new link is: https://huggingface.co/docs/datasets/process#data-augmentation
Issue: OverflowError when slicing with an array containing negative ids
State: closed

```python
from datasets import Dataset

d = ds.Dataset.from_dict({"a": range(10)})

print(d[0])
# {'a': 0}

print(d[-1])
# {'a': 9}

print(d[[0, -1]])
# OverflowError
```
results in
```
---------------------------------------------------------------------------
OverflowError                             Traceback (most recent call last)
<ipython-input-5-863dc3555598> in <module>
----> 1 d[[0, -1]]

~/Desktop/hf/nlp/src/datasets/arrow_dataset.py in __getitem__(self, key)
   1070             format_columns=self._format_columns,
   1071             output_all_columns=self._output_all_columns,
-> 1072             format_kwargs=self._format_kwargs,
   1073         )
   1074 

~/Desktop/hf/nlp/src/datasets/arrow_dataset.py in _getitem(self, key, format_type, format_columns, output_all_columns, format_kwargs)
   1025                 indices = key
   1026 
-> 1027             indices_array = pa.array([int(i) for i in indices], type=pa.uint64())
   1028 
   1029             # Check if we need to convert indices

~/.virtualenvs/hf-datasets/lib/python3.7/site-packages/pyarrow/array.pxi in pyarrow.lib.array()

~/.virtualenvs/hf-datasets/lib/python3.7/site-packages/pyarrow/array.pxi in pyarrow.lib._sequence_to_array()

OverflowError: can't convert negative value to unsigned int
```
Issue: Loss not decrease with Datasets and Transformers
State: closed

HI,

The following script is used to fine-tune a BertForSequenceClassification model on SST2.

The script is adapted from [this colab](https://colab.research.google.com/github/huggingface/datasets/blob/master/notebooks/Overview.ipynb) that presents an example of fine-tuning BertForQuestionAnswering using squad dataset. In that colab, loss works fine. When I adapt it to SST2, the loss fails to decrease as it should. I attach the adapted script below and appreciate anyone pointing out what I miss?

```python
import torch
from datasets import load_dataset
from transformers import BertForSequenceClassification
from transformers import BertTokenizerFast
# Load our training dataset and tokenizer
dataset = load_dataset("glue", 'sst2')
tokenizer = BertTokenizerFast.from_pretrained('bert-base-cased')
del dataset["test"] # let's remove it in this demo

# Tokenize our training dataset
def convert_to_features(example_batch):
    encodings = tokenizer(example_batch["sentence"])
    encodings.update({"labels": example_batch["label"]})
    return encodings

encoded_dataset = dataset.map(convert_to_features, batched=True)
# Format our dataset to outputs torch.Tensor to train a pytorch model
columns = ['input_ids', 'token_type_ids', 'attention_mask', 'labels']
encoded_dataset.set_format(type='torch', columns=columns)

# Instantiate a PyTorch Dataloader around our dataset
# Let's do dynamic batching (pad on the fly with our own collate_fn)
def collate_fn(examples):
    return tokenizer.pad(examples, return_tensors='pt')

dataloader = torch.utils.data.DataLoader(encoded_dataset['train'], collate_fn=collate_fn, batch_size=8)
# Now let's train our model
device = 'cuda' if torch.cuda.is_available() else 'cpu'
# Let's load a pretrained Bert model and a simple optimizer
model = BertForSequenceClassification.from_pretrained('bert-base-cased', return_dict=True)
optimizer = torch.optim.Adam(model.parameters(), lr=1e-5)
model.train().to(device)
for i, batch in enumerate(dataloader):
    batch.to(device)
    outputs = model(**batch)
    loss = outputs.loss
    loss.backward()
    optimizer.step()
    model.zero_grad()
    print(f'Step {i} - loss: {loss:.3}')


```
In case needed.

- datasets == 1.0.2
- transformers == 3.2.0

And I tested it on T5ForConditionalGeneration, that works no problem.

Hi did you manage to fix your issue ?

If so feel free to share your fix and close this thread
Issue: load_dataset from local squad.py, raise error: TypeError: 'NoneType' object is not callable 
State: closed


version: 1.0.2

```
train_dataset  = datasets.load_dataset('squad') 
```

The above code can works. However, when I download the squad.py from your server, and saved as `my_squad.py` to local. I run followings raise errors.
```
train_dataset  = datasets.load_dataset('./my_squad.py')                                                                                                
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-28-25a84b4d1581> in <module>
----> 1 train_dataset  = nlp.load_dataset('./my_squad.py')

/opt/conda/lib/python3.7/site-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, save_infos, script_version, **config_kwargs)
    602         hash=hash,
    603         features=features,
--> 604         **config_kwargs,
    605     )
    606 

TypeError: 'NoneType' object is not callable


Hi !
Thanks for reporting.
It looks like no object inherits from `datasets.GeneratorBasedBuilder` (or more generally from `datasets.DatasetBuilder`) in your script.

Could you check that there exist at least one dataset builder class ?

Hi @xixiaoyao did you manage to fix your issue ?

No activity, closing

It happened when try to change the old project which use 'nlp' to new project which use 'datasets'. You should check you old 'my_squad.py' file, change the inherit class from `nlp.xxx` to `datasets.xxx`. Otherwise datasets - load.py - import_main_class() `if inspect.isclass(obj) and issubclass(obj, main_cls_type):` can not find the main_cls.
Issue: Squad Metric Description & Feature Mismatch
State: closed

The [description](https://github.com/huggingface/datasets/blob/master/metrics/squad/squad.py#L39) doesn't mention `answer_start` in squad. However the `datasets.features` require [it](https://github.com/huggingface/datasets/blob/master/metrics/squad/squad.py#L68). It's also not used in the evaluation.

Thanks for reporting !
There indeed a mismatch between the features and the kwargs description

I believe `answer_start` was added to match the squad dataset format for consistency, even though it is not used in the metric computation. I think I'd rather keep it this way, so that you can just give `references=squad["answers"]` to `.compute()`.
Maybe we can just fix the description then.

But then providing the `answer_start` becomes mandatory since the format of the features is checked against the one provided in the squad [file](https://github.com/huggingface/datasets/pull/658/files).
Issue: Problem with JSON dataset format
State: open

I have a local json dataset with the following form.

{
    'id01234': {'key1': value1, 'key2': value2, 'key3': value3},
    'id01235': {'key1': value1, 'key2': value2, 'key3': value3},
    .
    .
    .
    'id09999': {'key1': value1, 'key2': value2, 'key3': value3}
}
Note that instead of a list of records it's basically a dictionary of key value pairs with the keys being the record_ids and the values being the corresponding record.

Reading this with json:

```
data = datasets.load('json', data_files='path_to_local.json')
```
Throws an error and asks me to chose a field. What's the right way to handle this?

Currently the `json` dataset doesn't support this format unfortunately.
However you could load it with
```python
from datasets import Dataset
import pandas as pd

df = pd.read_json("path_to_local.json", orient="index")
dataset = Dataset.from_pandas(df)
```

or you can make a custom dataset script as explained in doc here: https://huggingface.co/docs/datasets/add_dataset.html