t20062443/AT1ds huggingface dataset:
2dot71mily (CONTRIBUTOR) commented on 2020-12-10 20:55:07:
My dummy data seems quite large as a single row is composed of tokens/labels for an entire youtube video, with at least one row required for each file, which in this case 1 file per 13 youtube channels.

To make it smaller I passed `--n_lines 1` to reduce about 5x.

I then manually reduced size of the particularly long youtube lectures to get the size to about 30KB. However, after recompressing into a zip, and running dummy data test I got the following error:
`FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_youtube_caption_corrections - OSError: Cannot find data file. `, despite file being there, which I haven't had a chance yet to debug.
czabo (CONTRIBUTOR) commented on 2020-12-10 22:19:32:
> Nice thanks !
> 
> Can you rename the dataset folder and the dataset script name `hate_speech18` instead of `hate_speech_18` to follow the snake case convention we're using ?
> 
> Also it looks like the dummy_data.zip file is quite big (almost 4MB).
> Can you try to reduce its size ?
> 
> To do so feel free to take a look inside it and remove all the unnecessary files or chunks of texts. The idea is to only keep a few examples

Done, thanks!
anaerobeth (CONTRIBUTOR) commented on 2020-12-11 05:08:48:
@yjernite Thanks for the instructions. I manually added dummy data and created the zip file but one of the splits seem to return an empty list.

```
tests/test_dataset_common.py F                        [100%]

========================= FAILURES ==========================
_ LocalDatasetTest.test_load_dataset_all_configs_freebase_qa _

self = <tests.test_dataset_common.LocalDatasetTest testMethod=test_load_dataset_all_configs_freebase_qa>
dataset_name = 'freebase_qa'

    @slow
    def test_load_dataset_all_configs(self, dataset_name):
        configs = self.dataset_tester.load_all_configs(dataset_name, is_local=True)
>       self.dataset_tester.check_load_dataset(dataset_name, configs, is_local=True)

tests/test_dataset_common.py:237:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_dataset_common.py:198: in check_load_dataset
    self.parent.assertTrue(len(dataset[split]) > 0)
E   AssertionError: False is not true
```

Note that the dataset has `train`, `eval`, and `dev` (no test split). I am not sure if I am mapping them correctly when I called the Split Generator.
cahya-wirawan (CONTRIBUTOR) commented on 2020-12-11 08:17:51:
> Looks like there's a `Path` issue on windows. Could you try switching to
> `glob.glob(os.path.join(article_dir, "*.json"))`

Thanks, I replaced it with glob. Let's see if it solves the issue. Anyway, the main directory has a space, could it make the issue on windows? the test on linux don't have this problem.
cahya-wirawan (CONTRIBUTOR) commented on 2020-12-11 08:36:39:
It seems glob doesn't help also. Btw, one of the failing test tried to connect aws which failed:
```
C:\tools\miniconda3\lib\site-packages\urllib3\connection.py:160: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = ('s3.amazonaws.com', 443), timeout = 10, source_address = None
socket_options = [(6, 1, 1)]

```
Why did it try to connect to aws? I don't use it.
katnoria (CONTRIBUTOR) commented on 2020-12-11 08:42:33:
> it’s failing because of `RemoteDatasetTest.test_load_dataset_orange_sum`
> which i think is not the dataset you are doing a PR for. Try rebasing with:
> 
> ```
> git fetch upstream
> git rebase upstream/master
> git push -u -f origin your_branch
> ```

Thanks, my branch seems to be up to date. 
```Current branch add-wiki-lingua-dataset is up to date.```
hfawaz (CONTRIBUTOR) commented on 2020-12-11 09:54:34:
`amazon_polarity` is probably a subset of `amazon_us_reviews` but I am not entirely sure about that.
I guess `amazon_polarity` will help in reproducing results of papers using this dataset since even if it is a subset from `amazon_us_reviews`, it is not trivial how to extract `amazon_polarity` from `amazon_us_reviews`, especially since `amazon_us_reviews` was released after `amazon_polarity`.
SBrandeis (CONTRIBUTOR) commented on 2020-12-11 10:37:02:
Hello @hugoabonizio, thanks for the contribution.
Regarding the fake data, you can generate it manually.
Running the `python datasets-cli dummy_data datasets/offcombr` should give you instructions on how to manually create the dummy data.
For reference, here is a spec for `.arff` files : https://www.cs.waikato.ac.nz/ml/weka/arff.html
cahya-wirawan (CONTRIBUTOR) commented on 2020-12-11 11:15:21:
It seems that the circleci make a test for whole datasets repository, that means if only one of the dataset in the official repository has a download issue, this will also affect the test of a new dataset like mine, isn't it?
I changed the url to my newspaper dataset which contains only few simple json files and simple directory structure. But it still failed. And it failed not only on windows test. This is one of the error message:
```
-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_ajgt_twitter_ar
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_chr_en
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_ajgt_twitter_ar
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_chr_en
===== 4 failed, 2667 passed, 2052 skipped, 4 warnings in 432.05s (0:07:12) =====

Exited with code exit status 1
CircleCI received exit code 1
```
The test failed on twitter dataset even my dataset has nothing to do with twitter?
forest1988 (CONTRIBUTOR) commented on 2020-12-11 11:34:55:
Thank you for your detailed code review! It's so helpful.
I'll reflect them to the code in 24 hours.

You may have told me in Slack (I cannot find the conversation log though I've looked through threads), but I'm sorry it seems I'm still misunderstanding how to get YAML from the tagger.
I'm now asking on Slack if I am looking at the tagger the wrong way.
lhoestq (MEMBER) commented on 2020-12-11 13:39:51:
These errors
```
=========================== short test summary info ============================
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_ajgt_twitter_ar
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_chr_en
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_ajgt_twitter_ar
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_chr_en
```
appeared on master 3 hours ago and are now fixed.
(it was due to today's update of `xlrd` that broke two datasets)

You can ignore them, they're not related to your dataset
basakbuluz (CONTRIBUTOR) commented on 2020-12-11 13:45:06:
> These errors
> 
> ```
> =========================== short test summary info ============================
> FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_ajgt_twitter_ar
> FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_chr_en
> FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_ajgt_twitter_ar
> FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_chr_en
> ```
> 
> appeared on master 3 hours ago and are now fixed.
> (it was due to today's update of `xlrd` that broke two datasets)
> 
> You can ignore them, they're not related to your dataset

**If it is not a problem caused by us, we have already completed the review notes. You can then check it out and confirm?**
yjernite (MEMBER) commented on 2020-12-11 15:47:08:
Hi @purvimisal, thanks for your contributions!

This jokes dataset has come up before, and after a conversation with the initial submitter, we decided not to add it then. Humor is important, but looking at the actual data points in this set raises several concerns :) 

The main issue is the Reddit part of the dataset which has most of the examples. A cursory look at the data shows a large number of highly offensive jokes that reproduce some pretty harmful biases (the second one from the top is a Holocaust joke). 

The other two sources have similar issues (especially the "Blond Jokes") to a slightly lesser extent.

While such datasets can be useful in the right context, there is a real concern that people using the library might miss some of this context (however much we outline it), and end up unwittingly training models that rely on some pretty racist and sexist associations.

We would recommend skipping this dataset altogether.

If you feel really strongly about having a joke dataset, then we would ask that you:
- remove the Reddit part of the dataset altogether
- write an in-depth description of the social biases present in the remaining data

Let us know which of the two you decide! And if you want recommendations on other datasets to add, hit us up on Slack 🤗
lhoestq (MEMBER) commented on 2020-12-11 17:34:43:
Sure !

First please save your new dataset files somewhere.
Then you can do in this order:
```
git checkout master
git fetch upstream
git rebase upstream/master
git push
git checkout -b my-new-branch-name
```
This will create a new branch from the updated master branch.
Then you can re-add your files and commit + push them

Once it's done you should be able to create a new PR using your new branch :)
ZacharySBrown (CONTRIBUTOR) commented on 2020-12-11 22:20:02:
Hey @lhoestq , thanks for the feedback on this! I updated the `_generate_examples` with some comments on the process, and reduced the `dummy_data.zip` down quite a bit as well. 

For the dummy data, I reduced the text to only three sentences, and aligned the corresponding entity/token/sentence annotations to that (reduced accordingly). The frames file is a strange combined format for the annotations and I found if I reduced that that would break the parser no matter what I did, so I left that as is. The difference between a reduced frames and non-reduced frames file in the compressed dummy data was only about ~4kb, so hopefully leaving this as is will be ok!
rkc007 (CONTRIBUTOR) commented on 2020-12-11 22:40:00:
Thanks for the PR @iliemihai . 

Few comments - 

Can you run - 
`python datasets-cli dummy_data ./datasets/ronec --auto_generate` to generate dummy data.

Also, before committing files run : 
`make style`
`flake8 datasets`
then you can add and commit files.
iliemihai (CONTRIBUTOR) commented on 2020-12-11 23:03:39:
> Thanks for the PR @iliemihai .
> 
> Few comments -
> 
> Can you run -
> `python datasets-cli dummy_data ./datasets/ronec --auto_generate` to generate dummy data.
> 
> Also, before committing files run :
> `make style`
> `flake8 datasets`
> then you can add and commit files.

Sorry, forgot to generate dummy data. I will do it now :D
coltekin (NONE) commented on 2020-12-12 14:15:41:
Thank you for the heads up. I am not familiar with the terminology above (no idea what a sprint is), but I am happy that you found the data useful. Please feel free to distribute/use it as you see fit.

The OffensEval version you included in your data set has only binary labels. There is also a version [here](https://coltekin.github.io/offensive-turkish/troff-v1.0.tsv.gz) which also includes fine-grained labels similar to the OffensEval English data set - Just in case it would be of interest.

If you have questions about the data set, or need more information please let me know.
ncoop57 (NONE) commented on 2020-12-12 14:30:48:
@lhoestq Thanks for the review and suggestions! I've added your comments and pushed the changes. I'm having issues with the dummy data still. When I run the dummy data test

```bash
RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_so_stacksample
```
I get this error: 

```
___________________________________________ LocalDatasetTest.test_load_dataset_all_configs_so_stacksample ____________________________________________

self = <tests.test_dataset_common.LocalDatasetTest testMethod=test_load_dataset_all_configs_so_stacksample>, dataset_name = 'so_stacksample'

    @slow
    def test_load_dataset_all_configs(self, dataset_name):
        configs = self.dataset_tester.load_all_configs(dataset_name, is_local=True)
>       self.dataset_tester.check_load_dataset(dataset_name, configs, is_local=True)

tests/test_dataset_common.py:237: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_dataset_common.py:198: in check_load_dataset
    self.parent.assertTrue(len(dataset[split]) > 0)
E   AssertionError: False is not true

FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_so_stacksample - AssertionError: False is not true
```

I tried formatting the data similar to other datasets, but I think I don't have my csv's in the zip folder with the proper name. I also ran the command that's supposed to outline the exact steps I need to perform to get them into the correct format, but I followed them and they don't seem to be working still :/. Any help would be greatly appreciated!
cstorm125 (CONTRIBUTOR) commented on 2020-12-12 15:16:50:
> @lhoestq , I have removed the unnecessary files. Looks like I still have one error. How do I resolve this?

I think it's connection error on piqa dataset. Can you try triggering the test again? I usually resolve similar issues with:
```
git fetch upstream
git rebase upstream/master
git push -u -f origin your_branch_name
```
bharatr21 (CONTRIBUTOR) commented on 2020-12-12 18:11:21:
Thanks for the PR @MisbahKhan789 !

Few comments to help you along (I'm NOT a maintainer, just offering help to unblock the process) :-
 - Could you re-run `make style` and fix the errors related to code quality specific to your dataset in the `datasets/fake_news_english` folder?
(These seem to show errors that need manual fixes on running `flake8 datasets/fake_news_english`)
- Please run the local tests and check if they pass
`RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_real_dataset_<your-dataset-name>`
 `RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_<your-dataset-name>`
(If not, you may have to regenerate the dummy data and the `dataset_infos.json` files)
MisbahKhan789 (CONTRIBUTOR) commented on 2020-12-12 19:39:52:
> Thanks for the PR @MisbahKhan789 !
> 
> Few comments to help you along (I'm NOT a maintainer, just offering help to unblock the process) :-
> 
> * Could you re-run `make style` and fix the errors related to code quality specific to your dataset in the `datasets/fake_news_english` folder?
>   (These seem to show errors that need manual fixes on running `flake8 datasets/fake_news_english`)
> * Please run the local tests and check if they pass
>   `RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_real_dataset_<your-dataset-name>`
>   `RUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_<your-dataset-name>`
>   (If not, you may have to regenerate the dummy data and the `dataset_infos.json` files)

Hey Bharat, thanks for the reply. I actually submitted a new PR with the changes that are required :)
rkc007 (CONTRIBUTOR) commented on 2020-12-13 00:21:31:
@lhoestq Sorry for the mess. I don't know why this keeps on happening. I tried step by step process of updating the PR but seems something is wrong. This happened for 2nd time with the same PR. Apologies for that. 

New PR -> https://github.com/huggingface/datasets/pull/1527
Also, I fixed everything in the new PR.
yjernite (MEMBER) commented on 2020-12-13 17:23:43:
@rahul-art unfortunately this didn't quite work, here's how you can try again:
- `git checkout master` to go back to the main branch
- `git pull upstream master` to make it up to date
- `git checkout -b add_hind_encorp` to create a new branch

Then add the dataset script, `README.md`, `dummy_data.zip`, and `dataset_infos.json` to the tracked files for the branch with `git add` (please add all of these files individually, NOT the whole directory as we don't want the other data files)
Then after you have passed the style checks and the local tests, do:
- `git commit . -m initial_commit`
- `git push --set-upstream origin add_hind_encorp`

Then you can go to this branch on the WebApp and open a new PR
vrindaprabhu (CONTRIBUTOR) commented on 2020-12-13 19:09:54:
**NOTE:**

- Train and Dev: Manually downloaded (auto download is repeatedly giving `ConnectionError` for one of the files), Test: Auto Download
- Dummy test is passing
- The json file has been created with hard-coded paths for the manual downloads _(hardcoding has been removed from the final uploaded script)_
- datasets-cli is still **failing** . It is not picking the right directory for the config. For instance, my folder structure is as below:
 ```
            ~/Downloads/Data/
             |--- English-to-Polish
                   |--- (corresponding files) 
             |--- Russian-Polish
                   |--- (corresponding files) 
```

When ru-pl is selected, ideally it has to search in Russian-Polish folder, but it is searching in '/Downloads/Data/' folder and hence getting a FileNotFound error.

The command run is 
`python datasets-cli test datasets/poleval2019_mt/ --save_infos --all_configs --data_dir ~/Downloads/Data/
`
yjernite (MEMBER) commented on 2020-12-13 23:35:41:
Yes this has the right files!!!

I'll close the previous one then :) 

Now to pass the tests, you will need to:
- `make style` and run `flake8 datasets` from your repository root directory
- fix the dummy data

Did you generate the dummy data with the auto-generation tool (see the guide) or manually?
rahul-art (CONTRIBUTOR) commented on 2020-12-13 23:53:12:
yes I have give proper directory structure datasets/hind_encorp/dummy/0.0.0/dummy_data.zip  but in my dummy_data.zip only 1 file hind_encorp.plaintext is present because the dataset I got has only 1 file with both English and Hindi languages on 1 file itself may be this is causing issue
manandey (CONTRIBUTOR) commented on 2020-12-14 03:27:59:
> Before we merge can you try to reduce the size of the dummy_data.zip file ?
> 
> To do so feel free to only keep a few lines of the csv files ans also remove unnecessary chunks of texts (for example keep only the first sentences of a story).

Hi @lhoestq, I have reduced the size of the dummy_data.zip file by making the necessary changes you had suggested.
2dot71mily (CONTRIBUTOR) commented on 2020-12-14 05:39:28:
Sorry for delayed comments here. Last commit made two changes:
- Increased the valency of the labels from just True/False to more categories to describe the various types of diffs encountered. This required some rewrite of the README
- Reduced the number of remote files to be downloaded from 13 to 4, by combining all 13 of the channel-specific files together, and the splitting them up in a way to meet Github file size requirements. This also reduces size of the dummy-data.
lhoestq (MEMBER) commented on 2020-12-14 11:02:42:
You can just create another branch from master on your fork and create another PR:

first update your master branch
```
git checkout master
git fetch upstream
git rebase upstream/master
git push
```

then create a new branch
```
git checkout -b my-new-branch-name
```

Then you can add, commit and push the gnad10 files and open a new PR
lhoestq (MEMBER) commented on 2020-12-14 11:09:08:
The dummy json files must follow the exact same structure as the original json files.

However it looks like the dummy json files you have in your dummy_data.zip file are not structured the same way.
For example the original json is a dict with a field "Questions" that is a list of items.
However your dummy json is simply a list of items.

Can you update your dummy json files to follow the same structure ?
skyprince999 (CONTRIBUTOR) commented on 2020-12-14 12:49:51:
@lhoestq 
Updated the `dummy_data.zip `(<10kb)I had to reduce it to just a few samples. 
Train-Test-Dev  (20-5-5 samples) 

But the push also added changes from other PRs (probably because of a rebase!) So the files changed tab shows 466 files were changed!
thomwolf (MEMBER) commented on 2020-12-14 12:53:21:
I proposed something a few weeks ago in #898 (un-merged) but I think that the way that @mattbui added the dataset in the present PR is smarter and simpler should replace my PR #898.

(Narrator voice: *And it was around that time that Thomas realized that the community was now a lot smarter than him and he should hand-over the library he had started with @lhoestq to the community and stop pretending he knew everything about it.*)
yjernite (MEMBER) commented on 2020-12-14 13:03:07:
@ManuelFay thanks you so much for adding a dataset card, this is such a cool contribution!

This looks like it uses an old template for the card we've moved things around a bit and we have an app you should be using to get the tags and the structure of the Data Fields paragraph :) Would you mind moving your text to the newer format (we're also asking contributors to keep the full template structure, even if some sections still have [More Information Needed] for the time being)

Here's the link to the instructions:
https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#tag-the-dataset-and-write-the-dataset-card

Out of curiosity, what was your landing point for filling out the card? Did you follow the "Update on Github" when navigating the datasets? Trying to make the instructions as clear as possible :)
ManuelFay (CONTRIBUTOR) commented on 2020-12-14 14:25:46:
@yjernite 

Perfect, I'll follow the instructions when I have a bit more time tomorrow ! I was actually browsing the new contributions after the dataset sprint and realized most of the "old" datasets were not tagged, so I just copied and pasted the readme from another dataset and was not aware there was precise instructions... Will fix !

BTW, amazing job with the retriBert work, I used the contrastive + in-batch negative quite a bit for various projects. Probably neither the time nor place to talk about that but I was curious as to why, in your original work, you prefered using a simple projection in the last layer to differentiate the question vs answer embedding, rather than allowing for bias in the dense layer or even just to fine-tune 2 different embedders for question + answer ?
lhoestq (MEMBER) commented on 2020-12-14 14:49:00:
You can ignores these errors
```

=========================== short test summary info ===========================
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_ajgt_twitter_ar
FAILED tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_chr_en
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_ajgt_twitter_ar
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_chr_en
FAILED tests/test_dataset_common.py::RemoteDatasetTest::test_load_dataset_great_code
```

they're fixed on master
huu4ontocord (CONTRIBUTOR) commented on 2020-12-14 16:15:55:
Let me know why the pyarrow test is failing. For one of the config "trex", I had to load an initial datafile for a dictionary which is used to augment the rest of the datasets. In the dummy data, the dictionary file was truncated so I had to fudge that. I'm not sure if that is the issue.
lhoestq (MEMBER) commented on 2020-12-14 17:37:09:
Ok I found the issue with the dummy data.
It's currently failing because it's not generating a single example using the dummy csv file.
That's because there's only only line in the dummy csv file, and this line is skipped using the `next()` call used to ignore the headers of the csv.

To fix the dummy data you must add headers to the dummy csv files.