rIsHu009/github-issues-updated huggingface dataset:
Title: [Question & Bug Report] Can we preprocess a dataset on the fly?
Author: shuaihuaiyi (NONE)
State: closed
Created: 2021-01-24T09:28:24Z

I know we can use `Datasets.map` to preprocess a dataset, but I'm using it with very large corpus which generates huge cache file (several TB cache from a 400 GB text file). I have no disk large enough to save it.  Can we preprocess a dataset on the fly without generating cache?

BTW, I tried raising `writer_batch_size`. Seems that argument doesn't have any effect when it's larger than `batch_size`, because you are saving all the batch instantly after it's processed. Please check the following code:

https://github.com/huggingface/datasets/blob/0281f9d881f3a55c89aeaa642f1ba23444b64083/src/datasets/arrow_dataset.py#L1532

Comments:
We are very actively working on this. How does your dataset look like in practice (number/size/type of files)?

It's a text file with many lines (about 1B) of Chinese sentences. I use it to train language model using https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_mlm_wwm.py

Indeed I will submit a PR in a fez days to enable processing on-the-fly :)
This can be useful in language modeling for tokenization, padding etc.


any update on this issue? ...really look forward to use it 

Hi @acul3,

Please look at the discussion on a related Issue #1825. I think using `set_transform` after building from source should do.

@gchhablani thank you so much

will try look at it
Title: Efficient ways to iterate the dataset
Author: zhongpeixiang (CONTRIBUTOR)
State: closed
Created: 2021-01-24T07:54:31Z

For a large dataset that does not fits the memory, how can I select only a subset of features from each example?

If I iterate over the dataset and then select the subset of features one by one, the resulted memory usage will be huge. Any ways to solve this?

Thanks

Comments:
It seems that selecting a subset of colums directly from the dataset, i.e., dataset["column"], is slow.

I was wrong, ```dataset["column"]``` is fast.
Title: is it possible to make slice to be more compatible like python list and numpy?
Author: world2vec (NONE)
State: closed
Created: 2021-01-24T06:15:52Z

Hi,
see below error:
```
AssertionError: Requested slice [:10000000000000000] incompatible with 20 examples.
```

Comments:
Hi ! Thanks for reporting.
I am working on changes in the way data are sliced from arrow. I can probably fix your issue with the changes I'm doing.
If you have some code to reproduce the issue it would be nice so I can make sure that this case will be supported :)
I'll make a PR in a few days 

Good if you can take care at your side.
Here is the [colab notebook](https://colab.research.google.com/drive/19c-abm87RTRYgW9G1D8ktfwRW95zDYBZ?usp=sharing)
Title: Adding SICK dataset
Author: ghost (NONE)
State: closed
Created: 2021-01-24T02:15:31Z
Labels: dataset request

Hi
It would be great to include SICK dataset.

## Adding a Dataset
- **Name:** SICK
- **Description:** a well known entailment dataset 
- **Paper:** http://marcobaroni.org/composes/sick.html
- **Data:** http://marcobaroni.org/composes/sick.html
- **Motivation:** this is an important NLI benchmark

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).



thanks

Comments:
Title: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.2.1/datasets/csv/csv.py
Author: world2vec (NONE)
State: closed
Created: 2021-01-24T01:53:52Z

Hi,
When I load_dataset from local csv files, below error happened, looks raw.githubusercontent.com was blocked by the chinese government. But why it need to download csv.py? should it include when pip install the dataset?

```
Traceback (most recent call last):
  File "/home/tom/pyenv/pystory/lib/python3.6/site-packages/datasets/load.py", line 267, in prepare_module
    local_path = cached_path(file_path, download_config=download_config)
  File "/home/tom/pyenv/pystory/lib/python3.6/site-packages/datasets/utils/file_utils.py", line 343, in cached_path
    max_retries=download_config.max_retries,
  File "/home/tom/pyenv/pystory/lib/python3.6/site-packages/datasets/utils/file_utils.py", line 617, in get_from_cache
    raise ConnectionError("Couldn't reach {}".format(url))
ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.2.1/datasets/csv/csv.py

```

Comments:
I temporary manually download csv.py as custom dataset loading script

Indeed in 1.2.1 the script to process csv file is downloaded. Starting from the next release though we include the csv processing directly in the library.
See PR #1726 
We'll do a new release soon :)

Thanks.
Title: how can I combine 2 dataset with different/same features?
Author: world2vec (NONE)
State: closed
Created: 2021-01-24T01:26:06Z

to combine 2 dataset by one-one map like ds = zip(ds1, ds2):
ds1: {'text'}, ds2: {'text'}, combine ds:{'src', 'tgt'} 
or different feature:
ds1: {'src'}, ds2: {'tgt'}, combine ds:{'src', 'tgt'}

Comments:
Hi ! Currently we don't have a way to `zip` datasets but we plan to add this soon :)
For now you'll need to use `map` to add the fields from one dataset to the other. See the comment here for more info : https://github.com/huggingface/datasets/issues/853#issuecomment-727872188

Good to hear.
Currently I did not use map , just fetch src and tgt from the 2 dataset and merge them.
It will be a release if you can deal with it at the backend.
Thanks.

Hi! You can rename the columns and concatenate the datasets along `axis=1` to get the desired result as follows:
```python
ds1 = ds1.rename_column("text", "src")
ds2 = ds2.rename_column("text", "tgt")
ds = datasets.concatenate_datasets(["ds1", "ds2"], axis=1)
```
Title: Mention kwargs in the Dataset Formatting docs
Author: gchhablani (CONTRIBUTOR)
State: closed
Created: 2021-01-22T16:43:20Z

Hi,

This was discussed in Issue #1762 where the docs didn't mention that keyword arguments to `datasets.Dataset.set_format()` are allowed. 
To prevent people from having to check the code/method docs, I just added a couple of lines in the docs.

Please let me know your thoughts on this.

Thanks,
Gunjan

@lhoestq

Comments:
Title: Add Librispeech ASR
Author: patrickvonplaten (CONTRIBUTOR)
State: closed
Created: 2021-01-22T14:54:37Z

This PR adds the librispeech asr dataset: https://www.tensorflow.org/datasets/catalog/librispeech

There are 2 configs: "clean" and "other" whereas there are two "train" datasets for "clean", hence the name "train.100" and "train.360".

As suggested by @lhoestq, due to the enormous size of the dataset in `.arrow` format, the speech files are not directly prepared to a float32-array, but instead just the path to the array file is stored.

Comments:
> Awesome thank you !
> 
> The dummy data are quite big but it was expected given that the raw files are flac files.
> Given that the script doesn't even read the flac files I think we can remove them. Or maybe use empty flac files (see [here](https://hydrogenaud.io/index.php?topic=118685.0) for example). What do you think ?
> 
> We'll find a better solution to be able to have bigger dummy_data (max 1MB instead of a few KB, maybe using git LFS.

Hmm, I already made the dummy data as small as possible (a single flac filie per split only). I'd like to keep them at least to have complete dummy data and don't think 500KB for all datasets together is a problem (the long-range summarization datasets are similarly heavy). The moment we allow dummy data to be loaded directly for testing, we need the flac files IMO.

But I agree that longterm, we need a better solution for the dummy data (maybe stop hosting it on github to not make the repo too heavy)
Title: Issues when run two programs compute the same metrics
Author: lamthuy (NONE)
State: closed
Created: 2021-01-22T14:22:55Z

I got the following error when running two different programs that both compute sacreblue metrics. It seems that both read/and/write to the same location (.cache/huggingface/metrics/sacrebleu/default/default_experiment-1-0.arrow) where it caches the batches:

```
File "train_matching_min.py", line 160, in <module>ch_9_label
    avg_loss = valid(epoch, args.batch, args.validation, args.with_label)
  File "train_matching_min.py", line 93, in valid
    bleu += eval.compute()
  File "/u/tlhoang/projects/seal/match/models/eval.py", line 23, in compute
    return self.metric.compute()['score']
  File "/dccstor/know/anaconda3/lib/python3.7/site-packages/datasets/metric.py", line 387, in compute
    self._finalize()
  File "/dccstor/know/anaconda3/lib/python3.7/site-packages/datasets/metric.py", line 355, in _finalize
    self.data = Dataset(**reader.read_files([{"filename": f} for f in file_paths]))
  File "/dccstor/know/anaconda3/lib/python3.7/site-packages/datasets/arrow_reader.py", line 231, in read_files
    pa_table = self._read_files(files)
  File "/dccstor/know/anaconda3/lib/python3.7/site-packages/datasets/arrow_reader.py", line 170, in _read_files
    pa_table: pa.Table = self._get_dataset_from_filename(f_dict)
  File "/dccstor/know/anaconda3/lib/python3.7/site-packages/datasets/arrow_reader.py", line 299, in _get_dataset_from_filename
    pa_table = f.read_all()
  File "pyarrow/ipc.pxi", line 481, in pyarrow.lib.RecordBatchReader.read_all
  File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Expected to read 1819307375 metadata bytes, but only read 454396
```

Comments:
Hi ! To avoid collisions you can specify a `experiment_id` when instantiating your metric using `load_metric`. It will replace "default_experiment" with the experiment id that you provide in the arrow filename. 

Also when two `experiment_id` collide we're supposed to detect it using our locking mechanism. Not sure why it didn't work in your case. Could you share some code that reproduces the issue ? This would help us investigate.

Thank you for your response. I fixed the issue by set "keep_in_memory=True" when load_metric. 
I cannot share the entire source code but below is the wrapper I wrote:

```python
class Evaluation:
    def __init__(self, metric='sacrebleu'):
        # self.metric = load_metric(metric, keep_in_memory=True)
        self.metric = load_metric(metric)

    def add(self, predictions, references):
        self.metric.add_batch(predictions=predictions, references=references)

    def compute(self):
        return self.metric.compute()['score']
```

Then call the given wrapper as follows:

```python
eval = Evaluation(metric='sacrebleu')
for query, candidates,  labels in tqdm(dataset):
    predictions = net.generate(query)
    references = [[s] for s in labels]
    eval.add(predictions, references)
    if n % 100 == 0:
        bleu += eval.compute()
        eval = Evaluation(metric='sacrebleu')
Title: Connection Issues
Author: SaeedNajafi (NONE)
State: closed
Created: 2021-01-21T20:56:09Z

Today, I am getting connection issues while loading a dataset and the metric.
```
Traceback (most recent call last):
  File "src/train.py", line 180, in <module>
    train_dataset, dev_dataset, test_dataset = create_race_dataset()
  File "src/train.py", line 130, in create_race_dataset
    train_dataset = load_dataset("race", "all", split="train")
  File "/Users/saeed/Desktop/codes/repos/dreamscape-qa/env/lib/python3.7/site-packages/datasets/load.py", line 591, in load_dataset
    path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
  File "/Users/saeed/Desktop/codes/repos/dreamscape-qa/env/lib/python3.7/site-packages/datasets/load.py", line 267, in prepare_module
    local_path = cached_path(file_path, download_config=download_config)
  File "/Users/saeed/Desktop/codes/repos/dreamscape-qa/env/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 343, in cached_path
    max_retries=download_config.max_retries,
  File "/Users/saeed/Desktop/codes/repos/dreamscape-qa/env/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 617, in get_from_cache
    raise ConnectionError("Couldn't reach {}".format(url))
ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.2.1/datasets/race/race.py
```

Or

```
Traceback (most recent call last):
  File "src/train.py", line 105, in <module>
    rouge = datasets.load_metric("rouge")
  File "/Users/saeed/Desktop/codes/repos/dreamscape-qa/env/lib/python3.7/site-packages/datasets/load.py", line 500, in load_metric
    dataset=False,
  File "/Users/saeed/Desktop/codes/repos/dreamscape-qa/env/lib/python3.7/site-packages/datasets/load.py", line 267, in prepare_module
    local_path = cached_path(file_path, download_config=download_config)
  File "/Users/saeed/Desktop/codes/repos/dreamscape-qa/env/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 343, in cached_path
    max_retries=download_config.max_retries,
  File "/Users/saeed/Desktop/codes/repos/dreamscape-qa/env/lib/python3.7/site-packages/datasets/utils/file_utils.py", line 617, in get_from_cache
    raise ConnectionError("Couldn't reach {}".format(url))
ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.2.1/metrics/rouge/rouge.py
```

Comments:
Academic WIFI was blocking.
Title: PAWS-X: Fix csv Dictreader splitting data on quotes
Author: gowtham1997 (CONTRIBUTOR)
State: closed
Created: 2021-01-21T18:21:01Z

```python
from datasets import load_dataset
# load english paws-x dataset 
datasets = load_dataset('paws-x', 'en')
print(len(datasets['train']))                     # outputs 49202 but official dataset has 49401 pairs
print(datasets['train'].unique('label'))     # outputs [1, 0, -1] but labels are binary [0,1]
```

changed `data = csv.DictReader(f, delimiter="\t")` to `data = csv.DictReader(f, delimiter="\t", quoting=csv.QUOTE_NONE)` in the dataloader to make csv module not split by quotes.

The results are as expected for all languages after the change.

Comments:
Title: Unable to format dataset to CUDA Tensors
Author: gchhablani (CONTRIBUTOR)
State: closed
Created: 2021-01-21T15:31:23Z

Hi,

I came across this [link](https://huggingface.co/docs/datasets/torch_tensorflow.html) where the docs show show to convert a dataset to a particular format. I see that there is an option to convert it to tensors, but I don't see any option to convert it to CUDA tensors.

I tried this, but Dataset doesn't support assignment:
```
  columns=['input_ids', 'token_type_ids', 'attention_mask', 'start_positions','end_positions']

        samples.set_format(type='torch', columns = columns)
        for column in columns:
            samples[column].to(torch.device(self.config.device))
```
There should be an option to do so, or if there is already a way to do this, please let me know.

Thanks,
Gunjan

Comments:
Hi ! You can get CUDA tensors with

```python
dataset.set_format("torch", columns=columns, device="cuda")
```

Indeed `set_format` passes the `**kwargs` to `torch.tensor`

Hi @lhoestq,

Thanks a lot. Is this true for all format types?

As in, for 'torch', I can have `**kwargs` to `torch.tensor` and for 'tf' those args are passed to `tf.Tensor`, and the same for 'numpy' and 'pandas'?

Yes the keywords arguments are passed to the convert function like `np.array`, `torch.tensor` or `tensorflow.ragged.constant`.
We don't support the kwargs for pandas on the other hand.

Thanks @lhoestq,
Would it be okay if I added this to the docs and made a PR?

Sure ! Feel free to open a PR to improve the documentation :) 

Closing this issue as it has been resolved.
Title: Add SILICONE benchmark
Author: eusip (CONTRIBUTOR)
State: closed
Created: 2021-01-21T14:29:12Z

My collaborators and I within the Affective Computing team at Telecom Paris would like to re-submit our spoken dialogue dataset for publication.

This is a new pull request relative to the [previously closed request](https://github.com/huggingface/datasets/pull/1712) which was reviewed by @lhoestq.

Comments:
Thanks for the feedback. All your comments have been addressed!

Thank you for your constructive feedback! I now know how to best format future datasets that our team plans to publish in the near future :)

Awesome ! Looking forward to it :) 

Hi @lhoestq ! One last question. Our research team would like to distribute a link to this dataset amongst the spoken dialogue research community but the dataset does not show in the dropdown menu at huggingface.co. Is there anything else we must do in order to find the dataset there ?

Once the dataset does show in the dropdown menu, how can I affiliate it with the Telecom Paris organization that I already created at the website ?

The files are not located in the right place in the repo. Let me move them

I created a PR at https://github.com/huggingface/datasets/pull/1794

I just merged the change @eusip, now the dataset page is available at the url:
https://huggingface.co/datasets/silicone

Thank you for moving the folder for me :)
Title: More tags
Author: lhoestq (MEMBER)
State: closed
Created: 2021-01-21T13:50:10Z

Since the hub v2 is going to be released soon I figured it would be great to add the missing tags at least for some of the datasets of reference listed [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#write-the-loadingprocessing-code)

Comments:
Conll has `multilingual` but is only tagged as `en`

good catch, that was a bad copy paste x)
Title: wikipedia dataset incomplete
Author: ChrisDelClea (NONE)
State: closed
Created: 2021-01-21T11:47:15Z

Hey guys,

I am using the https://github.com/huggingface/datasets/tree/master/datasets/wikipedia dataset.
Unfortunately, I found out that there is an incompleteness for the German dataset.
 For reasons unknown to me, the number of inhabitants has been removed from many pages:
Thorey-sur-Ouche has 128 inhabitants according to the webpage (https://de.wikipedia.org/wiki/Thorey-sur-Ouche).
The pickle file however shows: französische Gemeinde mit  Einwohnern (Stand).
 Is it possible to fix this?

Best regards 
Chris

Comments:
Hi !
From what pickle file fo you get this ?
I guess you mean the dataset loaded using `load_dataset` ?

yes sorry, I used the `load_dataset`function and saved the data to a pickle file so I don't always have to reload it and are able to work offline. 

The wikipedia articles are processed using the `mwparserfromhell` library. Even if it works well in most cases, such issues can happen unfortunately. You can find the repo here: https://github.com/earwig/mwparserfromhell

There also exist other datasets based on wikipedia that were processed differently (and are often cleaner) such as `wiki40b`.



ok great. Thank you, @lhoestq.
Title: FewRel
Author: dspoka (NONE)
State: closed
Created: 2021-01-20T23:56:03Z
Labels: dataset request

## Adding a Dataset
- **Name:** FewRel
- **Description:**  Large-Scale Supervised Few-Shot Relation Classification Dataset
- **Paper:** @inproceedings{han2018fewrel,
               title={FewRel:A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation},
               author={Han, Xu and Zhu, Hao and Yu, Pengfei and Wang, Ziyun and Yao, Yuan and Liu, Zhiyuan and Sun, Maosong},
               booktitle={EMNLP},
               year={2018}}
- **Data:** https://github.com/ProKil/FewRel
- **Motivation:** relationship extraction dataset that's been used by some state of the art systems that should be incorporated.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

Comments:
+1

@dspoka Please check the following link : https://github.com/thunlp/FewRel
This link mentions two versions of the datasets. Also, this one seems to be the official link.

I am assuming this is the correct link and implementing based on the same.

Hi @lhoestq,

This issue can be closed, I guess.

Yes :) closing
Thanks again for adding FewRel !

Thanks for adding this @gchhablani ! Sorry didn't see the email notifications sooner!
Title: Ccaligned multilingual translation dataset
Author: flozi00 (CONTRIBUTOR)
State: closed
Created: 2021-01-20T22:18:44Z
Labels: dataset request

## Adding a Dataset
- **Name:** *name of the dataset*
- **Description:** *short description of the dataset (or link to social media or blog post)*
- CCAligned consists of parallel or comparable web-document pairs in 137 languages aligned with English. These web-document pairs were constructed by performing language identification on raw web-documents, and ensuring corresponding language codes were corresponding in the URLs of web documents. This pattern matching approach yielded more than 100 million aligned documents paired with English. Recognizing that each English document was often aligned to mulitple documents in different target language, we can join on English documents to obtain aligned documents that directly pair two non-English documents (e.g., Arabic-French).
- **Paper:** *link to the dataset paper if available*
- https://www.aclweb.org/anthology/2020.emnlp-main.480.pdf
- **Data:** *link to the Github repository or current dataset location*
- http://www.statmt.org/cc-aligned/
- **Motivation:** *what are some good reasons to have this dataset*
- The authors says it's an high quality dataset.
- it's pretty large and includes many language pairs. It could be interesting training mt5 on this task.

Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

Comments:
Title: Use a config id in the cache directory names for custom configs
Author: lhoestq (MEMBER)
State: closed
Created: 2021-01-20T11:11:00Z

As noticed by @JetRunner there was some issues when trying to generate a dataset using a custom config that is based on an existing config.

For example in the following code the `mnli_custom` would reuse the cache used to create `mnli` instead of generating a new dataset with the new label classes:
```python
from datasets import load_dataset

mnli = load_dataset("glue", "mnli")
mnli_custom = load_dataset("glue", "mnli", label_classes=["contradiction", "entailment", "neutral"])
```

I fixed that by extending the cache directory definition of a dataset that is being generated.
Instead of using the config name in the cache directory name, I switched to using a `config_id`.

By default it is equal to the config name.
However the name of a config is not sufficent to have a unique identifier for the dataset being generated since it doesn't take into account:
- the config kwargs that can be used to overwrite attributes
- the custom features used to write the dataset
- the data_files for json/text/csv/pandas datasets

Therefore the config id is just the config name with an optional suffix based on these.

In particular taking into account the config kwargs fixes the issue with the `label_classes` above.

I completed the current test cases by adding the case that was missing: overwriting an already existing config.

Comments:
Title: fix comet citations
Author: ricardorei (CONTRIBUTOR)
State: closed
Created: 2021-01-20T10:52:38Z

I realized COMET citations were not showing in the hugging face metrics page:

<img width="814" alt="Screenshot 2021-01-20 at 09 48 44" src="https://user-images.githubusercontent.com/17256847/105164848-8b9da900-5b0d-11eb-9e20-a38f559d2037.png">

This pull request is intended to fix that.

Thanks!

Comments:
Title: COMET metric citation
Author: ricardorei (CONTRIBUTOR)
State: closed
Created: 2021-01-20T09:54:43Z

In my last pull request to add COMET metric, the citations where not following the usual "format". Because of that they where not correctly displayed on the website: 

<img width="814" alt="Screenshot 2021-01-20 at 09 48 44" src="https://user-images.githubusercontent.com/17256847/105158000-686efb80-5b05-11eb-8bb0-9c85fdac2938.png">

This pull request is only intended to fix that.

Comments:
I think its better to create a new branch with this fix. I forgot I was still using the old branch.
Title: Updated README for the Social Bias Frames dataset
Author: mcmillanmajora (CONTRIBUTOR)
State: closed
Created: 2021-01-19T17:53:00Z

See the updated card at https://github.com/mcmillanmajora/datasets/tree/add-SBIC-card/datasets/social_bias_frames. I incorporated information from the [SBIC data statement](https://homes.cs.washington.edu/~msap/social-bias-frames/DATASTATEMENT.html), paper, and the corpus README file included with the dataset download.

Comments:
Title: Fix typo in README.md of cnn_dailymail
Author: forest1988 (CONTRIBUTOR)
State: closed
Created: 2021-01-19T03:06:05Z

When I read the README.md of `CNN/DailyMail Dataset`, there seems to be a typo `CCN`.

I am afraid this is a trivial matter, but I would like to make a suggestion for revision.

Comments:
Good catch, thanks!

Thank you for merging!
Title: Added metadata and correct splits for swda.
Author: gmihaila (CONTRIBUTOR)
State: closed
Created: 2021-01-18T18:36:32Z

Switchboard Dialog Act Corpus

I made some changes following  @bhavitvyamalik recommendation in #1678:

* Contains all metadata.
* Used official implementation from the [/swda](https://github.com/cgpotts/swda) repo.
* Add official train and test splits used in [Stolcke et al. (2000)](https://web.stanford.edu/~jurafsky/ws97) and validation split used in [Probabilistic-RNN-DA-Classifier](https://github.com/NathanDuran/Probabilistic-RNN-DA-Classifier).

Comments:
I will push updates tomorrow.

@lhoestq thank you for your comments! I went ahead and fixed the code 😃. Please let me know if I missed anything.
Title: datasets slicing with seed 
Author: ghost (NONE)
State: closed
Created: 2021-01-18T14:08:55Z

Hi
I need to slice a dataset with random seed, I looked into documentation here https://huggingface.co/docs/datasets/splits.html 
I could not find a seed option, could you assist me please how I can get a slice for different seeds?
thank you.
@lhoestq

Comments:
Hi :) 
The slicing API from https://huggingface.co/docs/datasets/splits.html doesn't shuffle the data.
You can shuffle and then take a subset of your dataset with
```python
# shuffle and take the first 100 examples
dataset = dataset.shuffle(seed=42).select(range(100))
```

You can find more information about shuffling and selecting rows in the documentation: https://huggingface.co/docs/datasets/processing.html#selecting-sorting-shuffling-splitting-rows

thank you so much

On Mon, Jan 18, 2021 at 3:17 PM Quentin Lhoest <notifications@github.com>
wrote:

> Hi :)
> The slicing API doesn't shuffle the data.
> You can shuffle and then take a subset of your dataset with
>
> # shuffle and take the first 100 examplesdataset = dataset.shuffle(seed=42).select(range(100))
>
> You can find more information about shuffling and selecting rows in the
> documentation:
> https://huggingface.co/docs/datasets/processing.html#selecting-sorting-shuffling-splitting-rows
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/datasets/issues/1747#issuecomment-762278134>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AM3GZM5D5MDPLJGI4IG3UADS2Q7GPANCNFSM4WHLOZJQ>
> .
>
Title: difference between wsc and wsc.fixed for superglue
Author: ghost (NONE)
State: closed
Created: 2021-01-18T00:50:19Z

Hi
I see two versions of wsc in superglue, and I am not sure what is the differences and which one is the original one. could you help to discuss the differences? thanks @lhoestq

Comments:
From the description given in the dataset script for `wsc.fixed`:
```
This version fixes issues where the spans are not actually substrings of the text.
```
Title: Add missing "brief" entries to reuters
Author: jbragg (CONTRIBUTOR)
State: closed
Created: 2021-01-17T07:58:49Z

This brings the number of examples for ModApte to match the stated `Training set (9,603 docs)...Test Set (3,299 docs)`

Comments:
@lhoestq I ran `make style` but CI code quality still failing and I don't have access to logs

It's also likely that due to the previous placement of the field initialization, much of the data about topics etc was simply wrong and carried over from previous entries. Model scores seem to improve significantly with this PR.
Title: Add GLUE Compat (compatible with transformers<3.5.0)
Author: JetRunner (CONTRIBUTOR)
State: closed
Created: 2021-01-17T05:54:25Z

Link to our discussion on Slack (HF internal)
https://huggingface.slack.com/archives/C014N4749J9/p1609668119337400

The next step is to add a compatible option in the new `run_glue.py`

I duplicated `glue` and made the following changes:
1. Change the name to `glue_compat`.
2. Change the label assignments for MNLI and AX.

Comments:
Maybe it would be simpler to just overwrite the order of the label classes of the `glue` dataset ?
```python
mnli = load_dataset("glue", "mnli", label_classes=["contradiction", "entailment", "neutral"])
```

Sounds good. Will close the issue if that works.
Title: error when run fine_tuning on text_classification
Author: XiaoYang66 (NONE)
State: closed
Created: 2021-01-16T02:23:19Z

dataset:sem_eval_2014_task_1
pretrained_model:bert-base-uncased

error description:
when i use these resoruce to train fine_tuning a text_classification on sem_eval_2014_task_1,there always be some problem(when i use other dataset ,there exist the error too). And i followed the colab code (url:https://colab.research.google.com/github/huggingface/notebooks/blob/master/examples/text_classification.ipynb#scrollTo=TlqNaB8jIrJW).


the error is like this :
`File "train.py", line 69, in <module>
    trainer.train()
  File "/home/projects/anaconda3/envs/calibration/lib/python3.7/site-packages/transformers/trainer.py", line 784, in train
    for step, inputs in enumerate(epoch_iterator):
  File "/home/projects/anaconda3/envs/calibration/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 435, in __next__
    data = self._next_data()
  File "/home/projects/anaconda3/envs/calibration/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 475, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/projects/anaconda3/envs/calibration/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/projects/anaconda3/envs/calibration/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
KeyError: 2`

this is my code :
```dataset_name = 'sem_eval_2014_task_1'
num_labels_size = 3
batch_size = 4
model_checkpoint = 'bert-base-uncased'
number_train_epoch = 5

def tokenize(batch):
return tokenizer(batch['premise'], batch['hypothesis'], truncation=True, )

def compute_metrics(pred):
labels = pred.label_ids
preds = pred.predictions.argmax(-1)
precision, recall, f1, _ = precision_recall_fscore_support(labels, preds, average='micro')
acc = accuracy_score(labels, preds)
return {
'accuracy': acc,
'f1': f1,
'precision': precision,
'recall': recall
}

model = BertForSequenceClassification.from_pretrained(model_checkpoint, num_labels=num_labels_size)
tokenizer = BertTokenizerFast.from_pretrained(model_checkpoint, use_fast=True)

train_dataset = load_dataset(dataset_name, split='train')
test_dataset = load_dataset(dataset_name, split='test')

train_encoded_dataset = train_dataset.map(tokenize, batched=True)
test_encoded_dataset = test_dataset.map(tokenize, batched=True)

args = TrainingArguments(
output_dir='./results',
evaluation_strategy="epoch",
learning_rate=2e-5,
per_device_train_batch_size=batch_size,
per_device_eval_batch_size=batch_size,
num_train_epochs=number_train_epoch,
weight_decay=0.01,
do_predict=True,
)
trainer = Trainer(
model=model,
args=args,
compute_metrics=compute_metrics,
train_dataset=train_encoded_dataset,
eval_dataset=test_encoded_dataset,
tokenizer=tokenizer
)

trainer.train()
trainer.evaluate()

Comments:
none
Title: fixes and improvements for the WebNLG loader
Author: Shimorina (CONTRIBUTOR)
State: closed
Created: 2021-01-15T21:45:23Z

- fixes test sets loading in v3.0
- adds additional fields for v3.0_ru
- adds info to the WebNLG data card

Comments:
The dataset card is fantastic!

Looks good to me! Did you check that this still passes the slow tests with the existing dummy data?

Yes, I ran and passed all the tests specified in [this guide](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#automatically-add-code-metadata), including the slow ones.

I just added the `from pathlib import Path` at the top to fix the script

I ran the tests locally and they all pass, merging

Thank you for the review!
Title: Conda support
Author: LysandreJik (MEMBER)
State: closed
Created: 2021-01-14T15:11:25Z

Will push a new version on anaconda cloud every time a tag starting with `v` is pushed (like `v1.2.2`).

Will appear here: https://anaconda.org/huggingface/datasets

Depends on `conda-forge` for now, so the following is required for installation:

```
conda install -c huggingface -c conda-forge datasets
```

Comments:
Nice thanks :) 
Note that in `datasets` the tags are simply the version without the `v`. For example `1.2.1`.

Do you push tags only for versions?

Yes I've always used tags only for versions
Title: Adjust BrWaC dataset features name
Author: jonatasgrosman (CONTRIBUTOR)
State: closed
Created: 2021-01-13T20:39:04Z

I added this dataset some days ago, and today I used it to train some models and realized that the names of the features aren't so good.

Looking at the current features hierarchy, we have "paragraphs" with a list of "sentences" with a list of "sentences?!". But the actual hierarchy is a "text" with a list of "paragraphs" with a list of "sentences".

I confused myself trying to use the dataset with these names. So I think it's better to change it.

Comments:
Title: Update add new dataset template
Author: sgugger (CONTRIBUTOR)
State: closed
Created: 2021-01-13T15:08:09Z

This PR fixes a few typos in the "Add new dataset template" and clarifies a bit what to do for the dummy data creation when the `auto_generate` flag can't work.

Comments:
Add new "dataset"? ;)

Lol, too used to Transformers ;-)
Title: connection issue with glue, what is the data url for glue? 
Author: ghost (NONE)
State: closed
Created: 2021-01-13T08:37:40Z

Hi
my codes sometimes fails due to connection issue with glue, could you tell me how I can have the URL datasets library is trying to read GLUE from to test the machines I am working on if there is an issue on my side or not
thanks

Comments:
Hello @juliahane, which config of GLUE causes you trouble?
The URLs are defined in the dataset script source code: https://github.com/huggingface/datasets/blob/master/datasets/glue/glue.py
Title: Couldn't reach swda.py
Author: yangp725 (NONE)
State: closed
Created: 2021-01-13T02:57:40Z

ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.2.0/datasets/swda/swda.py

Comments:
Hi @yangp725,
The SWDA has been added very recently and has not been released yet, thus it is not available in the `1.2.0` version of 🤗`datasets`.
You can still access it by installing the latest version of the library (master branch), by following instructions in [this issue](https://github.com/huggingface/datasets/issues/1641#issuecomment-751571471).
Let me know if this helps !

Thanks @SBrandeis ,
Problem solved by downloading and installing the latest version datasets.
Title: Is there support for Deep learning datasets?
Author: pablodz (NONE)
State: closed
Created: 2021-01-12T20:22:41Z

I looked around this repository and looking the datasets I think that there's no support for images-datasets. Or am I missing something? For example to add a repo like this https://github.com/DZPeru/fish-datasets

Comments:
Hi @ZurMaD!
Thanks for your interest in 🤗 `datasets`. Support for image datasets is at an early stage, with CIFAR-10 added in #1617 
MNIST is also on the way: #1730 

If you feel like adding another image dataset, I would advise starting by reading the [ADD_NEW_DATASET.md](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md) guide. New datasets are always very much appreciated 🚀
Title: Offline loading
Author: lhoestq (MEMBER)
State: closed
Created: 2021-01-12T15:21:57Z

As discussed in #824 it would be cool to make the library work in offline mode.
Currently if there's not internet connection then modules (datasets or metrics) that have already been loaded in the past can't be loaded and it raises a ConnectionError.
This is because `prepare_module` fetches online for the latest version of the module.

To make it work in offline mode one suggestion was to reload the latest local version of the module.
I implemented that and I also raise a warning saying that the module that is loaded is the latest local version.
```python
logger.warning(
    f"Using the latest cached version of the module from {cached_module_path} since it "
    f"couldn't be found locally at {input_path} or remotely ({error_type_that_prevented_reaching_out_remote_stuff})."
)
```

I added tests to make sure it works as expected and I needed to do a few changes in the code to be able to test things properly. In particular I added a parameter `hf_modules_cache` to `init_dynamic_modules` for testing purposes. It makes it possible to have temporary modules caches for testing.

I also added a `offline` context utility that allows to test part of the code by making all the requests fail as if there was no internet.

Close #824, close #761.

Comments:
It's maybe a bit annoying to add but could we maybe have as well a version of the local data loading scripts in the package?
The `text`, `json`, `csv`. Thinking about people like in #1725 who are expecting to be able to work with local data without downloading anything.

Maybe we can add them to package_data or something?

Yes I mentioned this in #824 as well. I'm looking into it

Alright now `csv`, `json`, `text` and `pandas` are "packaged datasets", i.e. they're part of the `datasets` package, which makes them available in offline mode without any change in terms of API:
```python
from datasets import load_dataset

d = load_dataset("csv", data_files=["path/to/data.csv"])
```

Instead of loading the dataset script from the module cache, it's loaded from inside the `datasets` package.

I updated the test to still be able to fetch the dummy data files for those datasets from `datasets/{text|csv|pandas|json}/dummy` in the repo.

Alright now all test pass :)
(I don't thank you windows)

LGTM! Since you're getting the local script's last modification date anyways do you think it might be a good idea to show it in the warning?

> LGTM! Since you're getting the local script's last modification date anyways do you think it might be a good idea to show it in the warning?

Yep good idea. I added the date in the warning. For example `(last modified on Mon Nov 30 11:01:56 2020)`
Title: load the local dataset
Author: xinjicong (NONE)
State: closed
Created: 2021-01-12T12:12:55Z

your guidebook's example is like
>>>from datasets import load_dataset
>>> dataset = load_dataset('json', data_files='my_file.json')
but the first arg is path...
so how should i do if i want to load the local dataset for model training?
i will be grateful if you can help me handle this problem!
thanks a lot!

Comments:
You should rephrase your question or give more examples and details on what you want to do.

it’s not possible to understand it and help you with only this information.

sorry for that.
i want to know how could i load the train set and the test set from the local ,which api or function should i use .


Did you try to follow the instructions in the documentation?
Here: https://huggingface.co/docs/datasets/loading_datasets.html#from-local-files

thanks a lot 
i find that the problem is i dont use vpn...
so i have to keep my net work even if i want to load the local data ?

We will solve this soon (cf #1724)

thanks a lot

Hi! `json` is a packaged dataset now, which means its script comes with the library and doesn't require an internet connection.
Title: could not run models on a offline server successfully
Author: lkcao (NONE)
State: closed
Created: 2021-01-12T06:08:06Z

Hi, I really need your help about this.
I am trying to fine-tuning a RoBERTa on a remote server, which is strictly banning internet. I try to install all the packages by hand and try to run run_mlm.py on the server. It works well on colab, but when I try to run it on this offline server, it shows:
![image](https://user-images.githubusercontent.com/49967236/104276256-25a88600-546a-11eb-9776-8ec695dfa24e.png)

is there anything I can do? Is it possible to download all the things in cache and upload it to the server? Please help me out...

Comments:
Transferred to `datasets` based on the stack trace.

Hi @lkcao !
Your issue is indeed related to `datasets`. In addition to installing the package manually, you will need to download the `text.py` script on your server. You'll find it (under `datasets/datasets/text`: https://github.com/huggingface/datasets/blob/master/datasets/text/text.py.
Then you can change the line 221 of `run_mlm_new.py` into:
```python
  datasets = load_dataset('/path/to/text.py', data_files=data_files)
```
Where `/path/to/text.py` is the path on the server where you saved the `text.py` script.

We're working on including the local dataset builders (csv, text, json etc.) directly in the `datasets` package so that they can be used offline

The local dataset builders (csv, text , json and pandas) are now part of the `datasets` package since #1726 :)
You can now use them offline
```python
datasets = load_dataset('text', data_files=data_files)
```

We'll do a new release soon

> The local dataset builders (csv, text , json and pandas) are now part of the `datasets` package since #1726 :)
> You can now use them offline
> 
> ```python
> datasets = load_dataset('text', data_files=data_files)
> ```
> 
> We'll do a new release soon

so the new version release now?

Yes it's been available since datasets 1.3.0 !
Title: [Scientific papers] Mirror datasets zip
Author: patrickvonplaten (CONTRIBUTOR)
State: closed
Created: 2021-01-12T01:15:40Z

Datasets were uploading to https://s3.amazonaws.com/datasets.huggingface.co/scientific_papers/1.1.1/arxiv-dataset.zip and https://s3.amazonaws.com/datasets.huggingface.co/scientific_papers/1.1.1/pubmed-dataset.zip respectively to escape google drive quota and enable faster download.

Comments:
> Nice !
> 
> Could you try to reduce the size of the dummy_data.zip files ? they're quite big (300KB)

Yes, I think it might make sense to enhance the tool a tiny bit to prevent this automatically

That's the lightest I can make it...it's long-range summarization so a single sample has ~11000 tokens. 

Ok thanks :)

Awesome good to merge for me :-)
Title: Fix column list comparison in transmit format
Author: lhoestq (MEMBER)
State: closed
Created: 2021-01-11T17:23:56Z

As noticed in #1718 the cache might not reload the cache files when new columns were added.
This is because of an issue in `transmit_format` where the column list comparison fails because the order was not deterministic. This causes the `transmit_format` to apply an unnecessary `set_format` transform with shuffled column names.

I fixed that by sorting the columns for the comparison and added a test.

To properly test that I added a third column `col_3` to the dummy_dataset used for tests.

Comments:
Title: SciFact dataset - minor changes
Author: dwadden (CONTRIBUTOR)
State: closed
Created: 2021-01-11T05:26:40Z

Hi,

SciFact dataset creator here. First of all, thanks for adding the dataset to Huggingface, much appreciated!

I'd like to make a few minor changes, including the citation information and the `_URL` from which to download the dataset. Can I submit a PR for this?

It also looks like the dataset is being downloaded directly from Huggingface's Google cloud account rather than via the `_URL` in [scifact.py](https://github.com/huggingface/datasets/blob/master/datasets/scifact/scifact.py). Can you help me update the version on gcloud?

Thanks,

Dave

Comments:
Hi Dave,
You are more than welcome to open a PR to make these changes! 🤗
You will find the relevant information about opening a PR in the [contributing guide](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md) and in the [dataset addition guide](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).

Pinging also @lhoestq for the Google cloud matter.

> I'd like to make a few minor changes, including the citation information and the `_URL` from which to download the dataset. Can I submit a PR for this?

Sure ! Also feel free to ping us for reviews or if we can help :)

> It also looks like the dataset is being downloaded directly from Huggingface's Google cloud account rather than via the `_URL` in [scifact.py](https://github.com/huggingface/datasets/blob/master/datasets/scifact/scifact.py). Can you help me update the version on gcloud?

What makes you think that ?
Afaik there's no scifact on our google storage




> > I'd like to make a few minor changes, including the citation information and the `_URL` from which to download the dataset. Can I submit a PR for this?
> 
> Sure ! Also feel free to ping us for reviews or if we can help :)
> 
OK! We're organizing a [shared task](https://sdproc.org/2021/sharedtasks.html#sciver) based on the dataset, and I made some updates and changed the download URL - so the current code points to a dead URL. I'll update appropriately once the task is finalized and make a PR.

> > It also looks like the dataset is being downloaded directly from Huggingface's Google cloud account rather than via the `_URL` in [scifact.py](https://github.com/huggingface/datasets/blob/master/datasets/scifact/scifact.py). Can you help me update the version on gcloud?
> 
> What makes you think that ?
> Afaik there's no scifact on our google storage

You're right, I had the data cached on my machine somewhere. 



I opened a PR about this: https://github.com/huggingface/datasets/pull/1780. Closing this issue, will continue there.