t20062443/AT1ds huggingface dataset:
lhoestq (MEMBER) commented on 2020-11-30 18:14:37:
> `dummy_data` right now contains all article files, keeping only the required articles for dummy data fails the dummy data test.
> Any idea ?

We should definitely find a way to make it work with only a few articles.

If it doesn't work right now for dummy data, I guess it's because it tries to load every single article file ?

If so, then maybe you can use `os.listdir` method to first check all the data files available in the path where the `articles.tgz` file is extracted. Then you can simply iter through the data files and depending on their ID, include them in the train or test set. With this method you should be able to have only a few articles files per split in the dummy data. Does that make sense ?
yjernite (MEMBER) commented on 2020-11-30 18:31:45:
This is a really good data card!!

Small changes to make it even better:
- Tags: the dataset has both "original" data and data that is "extended" from a source dataset: TydiQA - you should choose both options in the tagging apps
- The language and annotation creator tags are off: the language here is the questions: I understand it's a mix of crowd-sourced and expert-generated? Is there any machine translation involved? The annotations are the span selections: is that crowd-sourced?
- Personal and sensitive information: there should be a statement there, even if only to say that none could be found or that it only mentions public figures
zaidalyafeai (CONTRIBUTOR) commented on 2020-11-30 23:30:22:
> Hi @zaidalyafeai, really excited to get more Arabic coverage in the lib, thanks for your contribution!
> 
> Couple of last comments:
> 
> * this PR seems to modify some files that are unrelated to your dataset. Could you rebase from master? It should take care of that.
> * The dataset card is a good start! Can you describe the task in a few words and add more information in the Data Structure part, including listing and describing the fields? Also, if you don't know how to fill out a paragraph, or if you have some information but think more would be beneficial, please leave `[More Information Needed]` instead of `[N/A]`

I have no idea how some other files changed. I tried to rebase and push but this created some errors. I had to run the command 
`git push -u --force origin add-metrec-dataset` which might cause some problems.
reshinthadithyan (CONTRIBUTOR) commented on 2020-12-01 05:13:58:
> @reshinthadithyan that's a great start! You will also need to add a Dataset card, following the instructions given [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#manually-tag-the-dataset-and-write-the-dataset-card)

Hello @yjernite I'm facing issues in using the datasets-tagger Refer #1 in datasets-tagger. Thanks
lhoestq (MEMBER) commented on 2020-12-01 09:22:33:
Hi ! Sure that would be valuable to support .xz files. Feel free to open a separate PR for this.
And feel free to create the first test case for extracting compressed files if you have some inspiration (maybe create test_file_utils.py ?). We can still spend more time on tests next week when the sprint is over though so don't spend too much time on it.
lhoestq (MEMBER) commented on 2020-12-01 13:33:35:
Sorry you had to close your PR ! It looks like this week's sprint doesn't always make it easy to see what's being added/what's already added. 
Thank you for contributing to the library. You did a great job on adding LER so feel free to add other ones that you would like to see in the library, it will be a pleasure to review
yjernite (MEMBER) commented on 2020-12-01 14:03:22:
> > @reshinthadithyan that's a great start! You will also need to add a Dataset card, following the instructions given [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#manually-tag-the-dataset-and-write-the-dataset-card)
> 
> Hello @yjernite I'm facing issues in using the datasets-tagger Refer #1 in datasets-tagger. Thanks

Hi @reshinthadithyan ! Did you try with the latest version of the tagger? What issues are you facing?

We're also relaxed the dataset requirement for now, you'll only add to add the tags :)
lhoestq (MEMBER) commented on 2020-12-01 14:44:32:
Hi ! you can read the tests by logging in to circleci.

Anyway for information here are the errors : 
```
datasets/pib/pib.py:19:1: F401 'csv' imported but unused
datasets/pib/pib.py:20:1: F401 'json' imported but unused
datasets/pib/pib.py:36:84: W291 trailing whitespace
```
and 
```
FAILED tests/test_file_encoding.py::TestFileEncoding::test_no_encoding_on_file_open
```

To fix the `test_no_encoding_on_file_open` you just have to specify an encoding while opening a text file. For example `encoding="utf-8"`
lhoestq (MEMBER) commented on 2020-12-01 16:24:55:
If you've followed the instructions in here : https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#start-by-preparing-your-environment

(especially point 2. and the command `git remote add upstream ....`)

Then you can try
```
git checkout master
git fetch upstream
git rebase upstream/master
git checkout -b add-<my-new-dataset-name>
```
lhoestq (MEMBER) commented on 2020-12-01 17:28:44:
Indeed we only require the tags to be added now (the yaml part at the top of the dataset card).
Could you add them please ?
You can find more infos here : https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#tag-the-dataset-and-write-the-dataset-card
sileod (CONTRIBUTOR) commented on 2020-12-01 18:20:31:
> Really cool ! Thanks for adding this one :)
> Good job at adding all those citations for each task
> 
> Looks like the dummy data test doesn't pass. Maybe some files are missing in the dummy_data.zip files ?
> The error reports `pragmeval/verifiability/train.tsv` to be missing
> 
> Also could you add the tags part of the dataset card (the rest is optional) ?
> See more info here : https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#tag-the-dataset-and-write-the-dataset-card

In the prior commits I generated dataset_infos and the dummy files myself
Now they are generated with the cli, and the tests now seem to be passing better
I will look into the tag
JetRunner (CONTRIBUTOR) commented on 2020-12-01 18:25:58:
> LGTM thanks :)
> 
> 
> 
> Before we merge, could you add a dataset card ? see here for more info: https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#tag-the-dataset-and-write-the-dataset-card
> 
> 
> 
> Note that only the tags at the top of the dataset card are mandatory, if you feel like it's going to take too much time writing the rest to fill it all you can just skip the paragraphs

Nope. I don't think there is a citation. Also, can I do the dataset card later (maybe in bulk)?
lhoestq (MEMBER) commented on 2020-12-01 23:32:15:
Looks like you did a good job with dummy data in the first place !
The downside of automatically generated dummy data is that the files are heavier (here 40KB per file).
If you could replace the generated dummy files with the one you created yourself it would be awesome, since the one you did yourself are way lighter (around 1KB per file). Using small files make `git clone` run faster so we encourage to use small dummy_data files.
cstorm125 (CONTRIBUTOR) commented on 2020-12-02 01:14:41:
@lhoestq Thanks, Quentin. Removed the .ipynb_checkpoints and edited the README.md. The tests are failing because of other dataets. I'm figuring out why since the commits only have changes on `wisesight_sentiment`

```
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_class_flue
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_class_norwegian_ner
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_configs_flue
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_configs_norwegian_ner
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_flue
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_norwegian_ner
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_xglue
```
rabeehkarimimahabadi (NONE) commented on 2020-12-02 01:32:44:
here I share my dataloader currently for multiple tasks: https://gist.github.com/rabeehkarimimahabadi/39f9444a4fb6f53dcc4fca5d73bf8195 

I need to train my model distributedly with this dataloader, "MultiTasksataloader", currently this does not work in distributed fasion,
to save on memory I tried to use iterative datasets, could you have a look in this dataloader and tell me if this is indeed the case? not sure how to make datasets being iterative to not load them in memory, then I remove the sampler for dataloader, and shard the data per core, could you tell me please how I should implement this case in datasets library? and how do you find my implementation in terms of correctness? thanks
cstorm125 (CONTRIBUTOR) commented on 2020-12-02 01:39:46:
Test failing for same issues as https://github.com/huggingface/datasets/pull/939
Please advise.

```
=========================== short test summary info ============================
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_class_flue
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_class_norwegian_ner
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_configs_flue
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_configs_norwegian_ner
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_flue
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_norwegian_ner
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_xglue
===== 7 failed, 1309 passed, 932 skipped, 11 warnings in 166.71s (0:02:46) =====
```
cstorm125 (CONTRIBUTOR) commented on 2020-12-02 06:09:24:
Thank you for contributing a Thai dataset, @mapmeld ! I'm super hyped. 
One comment I may add is that wongnai-corpus has two datasets: review classification (this) and word tokenization (https://github.com/wongnai/wongnai-corpus/blob/master/search/labeled_queries_by_judges.txt).
Would it be possible for you to rename this one something along the line of `wongnai-reviews` so that when/if we include the word tokenization dataset, we will know which is which.

This helps solve my check_code_quality issue.
```
make style
black --line-length 119 --target-version py36 datasets/wongnai
flake8 datasets/wongnai
isort datasets/wongnai/wongnai.py
```
mapmeld (CONTRIBUTOR) commented on 2020-12-02 06:38:33:
@cstorm125 thanks! following your suggestions on formatting and on naming the dataset

I am writing a blog post about Thai NLP and transformers (example: mBERT does 1-2 character tokens instead of doing word segmentation), started adding this dataset to use as an example, and then saw you were adding other datasets. Great work! And if you know any Thai BERT models beyond https://github.com/ThAIKeras/bert we should maybe talk over email!
rabeehk (CONTRIBUTOR) commented on 2020-12-02 08:50:34:
my implementation of sharding per TPU core: https://github.com/google-research/ruse/blob/d4dd58a2d8efe0ffb1a9e9e77e3228d6824d3c3c/seq2seq/trainers/t5_trainer.py#L316 
my implementation of dataloader for this case https://github.com/google-research/ruse/blob/d4dd58a2d8efe0ffb1a9e9e77e3228d6824d3c3c/seq2seq/tasks/tasks.py#L496
bharatr21 (CONTRIBUTOR) commented on 2020-12-02 09:15:10:
CI failures seem to be unrelated (related to `norwegian_ner`)
```
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_class_norwegian_ner
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_configs_norwegian_ner
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_norwegian_ner
```
lhoestq (MEMBER) commented on 2020-12-02 10:10:54:
Indeed the error reports that `pragmeval/verifiability/train.tsv` is missing for the verifiability dummy_data.zip file.
To fix that you should add the missing data files in each dummy_data.zip file.
To test that your dummy data work you can run
```
RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_
```
if some file is missing it should tell you which one
arc-bu (NONE) commented on 2020-12-02 11:04:21:
I see the issue happening again today - 

[nltk_data] Downloading package stopwords to /root/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
Downloading and preparing dataset cnn_dailymail/3.0.0 (download: 558.32 MiB, generated: 1.28 GiB, post-processed: Unknown size, total: 1.82 GiB) to /root/.cache/huggingface/datasets/cnn_dailymail/3.0.0/3.0.0/0128610a44e10f25b4af6689441c72af86205282d26399642f7db38fa7535602...

---------------------------------------------------------------------------

NotADirectoryError                        Traceback (most recent call last)

<ipython-input-9-cd4bf8bea840> in <module>()
     22 
     23 
---> 24 train = load_dataset('cnn_dailymail', '3.0.0', split='train')
     25 validation = load_dataset('cnn_dailymail', '3.0.0', split='validation')
     26 test = load_dataset('cnn_dailymail', '3.0.0', split='test')

5 frames

/root/.cache/huggingface/modules/datasets_modules/datasets/cnn_dailymail/0128610a44e10f25b4af6689441c72af86205282d26399642f7db38fa7535602/cnn_dailymail.py in _find_files(dl_paths, publisher, url_dict)
    132     else:
    133         logging.fatal("Unsupported publisher: %s", publisher)
--> 134     files = sorted(os.listdir(top_dir))
    135 
    136     ret_files = []

NotADirectoryError: [Errno 20] Not a directory: '/root/.cache/huggingface/datasets/downloads/1bc05d24fa6dda2468e83a73cf6dc207226e01e3c48a507ea716dc0421da583b/cnn/stories'

Can someone please take a look ?
lhoestq (MEMBER) commented on 2020-12-02 18:14:34:
Thanks ! The github diff looks all clean now :) 
To fix the CI you just need to rebase from master

Don't forget to add the tags of the dataset card. It's the yaml part at the top of the dataset card
More infor here : https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#tag-the-dataset-and-write-the-dataset-card

The issue you had with the tagger should be fixed now by https://github.com/huggingface/datasets-tagging/pull/5
bharatr21 (CONTRIBUTOR) commented on 2020-12-02 18:29:45:
> If the CI fails just because of `RemoteDatasetTest` errors it's fine, they're fixed on master

The same 3 tests are failing again :(
```
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_class_norwegian_ner
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_builder_configs_norwegian_ner
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_norwegian_ner
```
bharatr21 (CONTRIBUTOR) commented on 2020-12-02 18:35:08:
> One trick if you want to add more datasets to avoid these errors : you can just rebase the master branch of your fork from the master branch of the repo. Then each time you make a new branch from master on your fork, it will include the fix for these errors

Yes, I almost always do that, but somehow seems even this branch got old 😓 
I also do the following if I directly create a new branch locally: `git checkout -b <branchname> upstream/master` so it stays up-to date irrespective of my fork, still don't know how this crept in again
madlag (CONTRIBUTOR) commented on 2020-12-02 22:49:42:
Yes, sorry I did not see earlier your message. I added 34 on the 35 datasets in CodeXGlue, tomorrow I will wrap it up, and so I will remove my version for code_refinement. Maybe we can just have a renaming after the merge, to have a consistent naming with all the other codexglue datasets ? What do you think @reshinthadithyan ?
reshinthadithyan (CONTRIBUTOR) commented on 2020-12-03 03:21:43:
> Yes, sorry I did not see earlier your message. I added 34 on the 35 datasets in CodeXGlue, tomorrow I will wrap it up, and so I will remove my version for code_refinement. Maybe we can just have a renaming after the merge, to have a consistent naming with all the other codexglue datasets ? What do you think @reshinthadithyan ?

Hello @madlag, I think you can retain that in your script. Let's stick onto the same file like how Glue is maintained.
leoxzhao (CONTRIBUTOR) commented on 2020-12-03 15:49:21:
Also, dummy_data seems having difficulty to handle manually downloaded datasets. `python datasets-cli dummy_data datasets/ms_terms  --data_dir ...` reported `error: unrecognized arguments: --data_dir` error. Without `--data_dir`, it reported this error:
```
Dataset ms_terms with config BuilderConfig(name='ms_terms-full', version=1.0.0, data_dir=None, data_files=None, description='...\n') seems to already open files in the method `_split_generators(...)`. You might consider to instead only open files in the method `_generate_examples(...)` instead. If this is not possible the dummy data has to be created with less guidance. Make sure you create the file None.
Traceback (most recent call last):
  File "datasets-cli", line 36, in <module>
    service.run()
  File "/Users/lzhao/Downloads/huggingface/datasets/src/datasets/commands/dummy_data.py", line 326, in run
    dataset_builder=dataset_builder, mock_dl_manager=mock_dl_manager
  File "/Users/lzhao/Downloads/huggingface/datasets/src/datasets/commands/dummy_data.py", line 406, in _print_dummy_data_instructions
    for split in generator_splits:
UnboundLocalError: local variable 'generator_splits' referenced before assignment
```
lhoestq (MEMBER) commented on 2020-12-03 16:26:35:
also pinging @thomwolf 
The dataset webpage returns a form when trying to download the dataset (form here : http://www.cfilt.iitb.ac.in/iitb_parallel/dataset.html).
However the url we get with the form can be used for the dataset script.
Should we ask the authors or use the urls this way ?
lhoestq (MEMBER) commented on 2020-12-03 19:01:55:
Oh yes `--data_dir` seems to only be supported for the `datasets_cli test` command. Sorry about that.

Can you try to build the dummy_data.zip file manually ?

It has to be inside `./datasets/ms_terms/dummy/ms_terms-full/1.0.0`.
Inside this folder, please create a folder `dummy_data` that contains a dummy file `MicrosoftTermCollection.tbx` (with just a few examples in it). Then you can zip the `dummy_data` folder to `dummy_data.zip`

Then you can check if it worked using the command
```
pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_ms_terms
```

Feel free to use some debugging print statements in your script if it doesn't work first try to see what `dl_manager.manual_dir` ends up being and also `path_to_manual_file`.

Feel free to ping me if you have other questions
leoxzhao (CONTRIBUTOR) commented on 2020-12-04 01:29:44:
`pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_ms_terms` gave `1 passed, 4 warnings in 8.13s`. Existing datasets, like `wikihow`, and `newsroom`, also report 4 warnings. So, I guess that is not related to this dataset.
spatil6 (CONTRIBUTOR) commented on 2020-12-04 05:55:18:
> also pinging @thomwolf
> The dataset webpage returns a form when trying to download the dataset (form here : http://www.cfilt.iitb.ac.in/iitb_parallel/dataset.html).
> However the url we get with the form can be used for the dataset script.
> Should we ask the authors or use the urls this way ?

I had discussion on this with @thomwolf . We have already sent email to author of this dataset.
mrshu (NONE) commented on 2020-12-04 08:22:39:
I am sorry for hijacking this closed issue, but I believe I was able to reproduce this very issue. Strangely enough, it also turned out that running `pip install "pyarrow<1" --upgrade` did indeed fix the issue (PyArrow was installed in version `0.14.1` in my case).

Please see the Colab below:

https://colab.research.google.com/drive/15QQS3xWjlKW2aK0J74eEcRFuhXUddUST

Thanks!
thomwolf (MEMBER) commented on 2020-12-04 08:55:10:
Hey @sileod! Super nice to see you participating ;)

Did you officially joined the sprint by posting on [the forum thread](https://discuss.huggingface.co/t/open-to-the-community-one-week-team-effort-to-reach-v2-0-of-hf-datasets-library/2176) and joining our slack?

I can't seem to find you there! Should I add you directly with your gmail address?
thomwolf (MEMBER) commented on 2020-12-04 08:58:11:
Hey @zhongpeixiang!
Really nice addition here!

Did you officially joined the sprint by posting [on the forum thread](https://discuss.huggingface.co/t/open-to-the-community-one-week-team-effort-to-reach-v2-0-of-hf-datasets-library/2176) and joining our slack?
I can't seem to find you there! Should I add you directly with your gmail address?
zhongpeixiang (CONTRIBUTOR) commented on 2020-12-04 09:57:45:
> Hey @zhongpeixiang!
> Really nice addition here!
> 
> Did you officially joined the sprint by posting [on the forum thread](https://discuss.huggingface.co/t/open-to-the-community-one-week-team-effort-to-reach-v2-0-of-hf-datasets-library/2176) and joining our slack?
> I can't seem to find you there! Should I add you directly with your gmail address?

Thank you for the invitation. This initiative is awesome. Sadly I’m occupied by my thesis writing this month. Good luck 🤗
zhongpeixiang (CONTRIBUTOR) commented on 2020-12-04 10:58:19:
> As you want @zhongpeixiang (I was maybe not clear but that just mean that by posting on the forum thread that you participated in the current event you will get a special gift (a tee-shirt) for the contribution that you have already done here :-) Nothing more to do)

Oh, I misunderstood the post. I'm glad to join.
yuxiang-wu (CONTRIBUTOR) commented on 2020-12-04 18:21:26:
> Thanks ! could you regenerate the dataset_infos.json file ?
> 
> ```
> datasets-cli test ./datasets/mrqa --save_infos --all_configs --ignore_verifications
> ```
> 
> also cc @VictorSanh

Updated the `dataset_infos.json` file.
mattbui (CONTRIBUTOR) commented on 2020-12-05 04:37:44:
@lhoestq so I ended up removing all the nested JSON objects in the gz datafile and keep only one object with minimal content: `{"publication_number": "US-8230922-B2", "abstract": "dummy abstract", "application_number": "US-201113163519-A", "description": "dummy description"}`. 

They're reduced to 35KB in total (2.5KB per domain and 17.5KB for all domains), hopefully, they're small enough.
yjernite (MEMBER) commented on 2020-12-05 20:14:09:
Cool dataset! Replying in-line:

> This PR adds the **ReFreSD dataset**.
> The original data is hosted [on this github repo](https://github.com/Elbria/xling-SemDiv) and we use the `REFreSD_rationale` to expose all the data.
> 
> Need feedback on:
> 
> * I couldn't generate the dummy data. The file we download is a tsv file, but without extension, I suppose this is the problem. I'm sure there is a simple trick to make this work.

you can use `--match_text_files` in the dummy data generation:
`python datasets-cli dummy_data datasets/refresd --auto_generate --match_text_files "REFreSD_rationale"`

> * The feature names.
>   
>   * I don't know if it's better to stick to the classic `sentence1`, `sentence2` or to `sentence_en`, `sentence_fr` to be more explicit.

It would actually be even better to use the `Translation` feature here to replace best:
`"sentence_pair": datasets.Translation(languages=['en', 'fr']),`

Then during `_generate_examples` this filed should look like"
`{"sentence_pair": {"fr": french, "en": english}}`

>   * There is a binary label (called `label`, no problem here), and a 3-class label called `#3_labels` in the original tsv. I changed it to `all_labels` but I'm sure there is better.
Looks good!

> * The rationales are lists of integers, extracted as a string at first. I wonder what's the best way to treat them, any idea? Also, I couldn't manage to make a `Sequence` of `int8` but I'm sure I've missed something simple.

Having the feature declared as `"rationale_en": datasets.Sequence(datasets.Value("int32"))` should work

> 
> Thanks in advance

Hope that helps you out! Don't forget to `make style`, rebase from master, and run all the tests before pushing again! You will also need to add a `README.md` as described in the guide:
https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#tag-the-dataset-and-write-the-dataset-card