Title: Transfer some canonical datasets under an organization namespace
Author: albertvillanova (MEMBER)
State: closed
Created: 2022-10-10T15:44:31Z
Labels: dataset contribution

As discussed during our @huggingface/datasets meeting, we are planning to move some "canonical" dataset scripts under their corresponding organization namespace (if this does not exist).

On the contrary, if the dataset already exists under the organization namespace, we are deprecating the canonical one (and eventually delete it).

First, we should test it using a dummy dataset/organization.

TODO:
- [x] Test with a dummy dataset
  - [x] Create dummy canonical dataset: https://huggingface.co/datasets/dummy_canonical_dataset
  - [x] Create dummy organization: https://huggingface.co/dummy-canonical-org
  - [x] Transfer dummy canonical dataset to dummy organization
- [ ] Transfer datasets
  - [x] babi_qa => facebook
  - [x] blbooks => TheBritishLibrary/blbooks
  - [x] blbooksgenre => TheBritishLibrary/blbooksgenre
  - [x] common_gen => allenai
  - [x] commonsense_qa => tau
  - [x] competition_math => hendrycks/competition_math
  - [x] cord19 => allenai
  - [x] emotion => dair-ai
  - [ ] gem => GEM
  - [x] hellaswag => Rowan
  - [x] hendrycks_test => cais/mmlu
  - [x] indonlu => indonlp
  - [ ] multilingual_librispeech => facebook
    - It already exists "facebook/multilingual_librispeech"
  - [ ] oscar => oscar-corpus
  - [x] peer_read => allenai
  - [x] qasper => allenai
  - [x] reddit => webis/tldr-17
  - [x] russian_super_glue => russiannlp
  - [x] rvl_cdip => aharley
  - [x] s2orc => allenai
  - [x] scicite => allenai
  - [x] scifact => allenai
  - [x] scitldr => allenai
  - [x] swiss_judgment_prediction => rcds
  - [x] the_pile => EleutherAI
  - [ ] wmt14, wmt15, wmt16, wmt17, wmt18, wmt19,... => wmt
- [ ] Deprecate (and eventually remove) datasets that cannot be transferred because they already exist
  - [x] banking77 => PolyAI
  - [x] common_voice => mozilla-foundation
  - [x] german_legal_entity_recognition => elenanereiss
- ... 

EDIT: the list above is continuously being updated

Comments:
The transfer of the dummy dataset to the dummy org works as expected:
```python
In [1]: from datasets import load_dataset; ds = load_dataset("dummy_canonical_dataset", download_mode="force_redownload"); ds
Downloading builder script: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.98k/2.98k [00:00<00:00, 2.01MB/s]
Downloading and preparing dataset dummy_canonical_dataset/default (download: 411 bytes, generated: 385 bytes, post-processed: Unknown size, total: 796 bytes) to .../.cache/huggingface/datasets/dummy_canonical_dataset/default/1.0.0/100870c358637e269fee140585e61e1472d5075a9bf6f866719934c725e55fb4...
Downloading data: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 411/411 [00:00<00:00, 293kB/s]
Dataset dummy_canonical_dataset downloaded and prepared to .../.cache/huggingface/datasets/dummy_canonical_dataset/default/1.0.0/100870c358637e269fee140585e61e1472d5075a9bf6f866719934c725e55fb4. Subsequent calls will reuse this data.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 304.16it/s]
Out[1]: 
DatasetDict({
    train: Dataset({
        features: ['langs', 'ner_tags', 'tokens'],
        num_rows: 3
    })
})

In [2]: from datasets import load_dataset; ds = load_dataset("dummy-canonical-org/dummy_canonical_dataset"); ds
Downloading builder script: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.98k/2.98k [00:00<00:00, 1.57MB/s]
Downloading and preparing dataset dummy_canonical_dataset/default to .../.cache/huggingface/datasets/dummy-canonical-org___dummy_canonical_dataset/default/1.0.0/100870c358637e269fee140585e61e1472d5075a9bf6f866719934c725e55fb4...
Dataset dummy_canonical_dataset downloaded and prepared to .../.cache/huggingface/datasets/dummy-canonical-org___dummy_canonical_dataset/default/1.0.0/100870c358637e269fee140585e61e1472d5075a9bf6f866719934c725e55fb4. Subsequent calls will reuse this data.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 362.48it/s]
Out[2]: 
DatasetDict({
    train: Dataset({
        features: ['langs', 'ner_tags', 'tokens'],
        num_rows: 3
    })
})
```

Cool ! 🚀 

Maybe we should be a bit more proactive with these transfers. There are only ≈70 canonical models, so reaching that number with datasets would be great, too. It's not easy considering the current number of ≈750 canonical datasets, but doable.

For instance, it shouldn't be too hard to transfer these datasets (partial list; all of them have more than > 1k downloads):

<details>

<summary> Datasets to transfer </summary>

```
quickdraw -> google
openai_humaneval -> openai
c4 -> allenai/c4 (the canonical version reads data from the org version)
mbpp -> google (ask jaaustin (author) where to transfer the dataset)
competition_math -> hendrycks (author)
gsm8k -> openai
ai2_arc -> allenai
imdb -> stanfordai
greek_legal_code -> chrispap (author)
spider -> Yale-LILY
squad and squad_v2 -> rajpurkarlab (or rajpurkar, a member of the org and one of the authors)
cppe-5 -> rishitdagli
news_commentary -> Helsinki-NLP
jfleg -> keisks (author)
pubmed_qa -> qiaojin (author)
medmcqa -> infinitylogesh (author)
cifar10 and cifar100 -> UniversityofToronto
cc100 -> gwenzek (author)
asset -> facebook
blbooks -> BritishLibraryLabs
capes -> FLSRDS (maybe the author?)
cc_news -> fhamborg (author)
clue -> CLUE benchmark
coqa -> stanfordnlp
lambada -> germank (author)
librispeech_asr -> openslr
drop -> allenai
duorc -> salesforce (ask amritasaha87 (author) where to transfer)
glue -> nyu-mll  ?
go_emotions -> google
commonsense_qa -> tau
dbpedia_14 -> JensLehmann (author?)
discofuse -> google
mc4 -> allenai/c4
openbookqa -> allenai
ropes -> allene
trivia_qa -> mandarjoshi (author)
wikiann -> afshinrahimi (author)
xtreme -> google
xscr -> INK-USC
yelp_review_full -> Yelp
truthful_qa -> jacobhilton22 (author)
bigbench -> google
xnli -> facebook
sciq -> allenai
sst2 -> stanfordnlp
blimp -> alexwarstadt (author)
tweet_eval -> cardiffnlp
beans -> AI-Lab-Makerere
lex_glue -> coastalcph
americas_nli -> abteen (author)
opus_euconst -> tiedeman (author)
medical_questions_pairs -> curaihealth
web_questions -> joberant (author)
anli -> facebook
race -> CarnegieMellonCS
klue -> klue
wino_bias -> uclanlp
wiki_qa -> microsoft
xcopa -> cambridgeltl
indic_glue -> ai4bharat
boolq -> google
adversarial_qa -> mbartolo (author)
nq_open -> google
snli -> stanfordnlp
stsb_multi_mt -> PhilipMay (author)
multi_nli -> sleepinyourhat (author)
paws -> google
paws-x -> google
ms_marco - microsoft
xquad -> deepmind
narrativeqa -> deepmind
kilt_tasks -> facebook
hate_speech_offensive -> tdavidson (author)
wiki40b -> google
covost2 -> facebook
common_gen -> INKLAB
multi_eurlex -> kiddothe2b (author)
exams -> mhardalov (author)
tiny_shakespeare -> karpathy (author)
blbooksgenre -> BritishLibraryLabs  ?
food101 -> ethz  ?
scitail -> allenai
billsum -> FiscalNote
imppres -> facebook
quartz -> allenai
qasc -> allenai
quail -> textmachinelab
wiki_lingua -> esdurmus
cos_e -> salesforce  ?
civil_comments -> google  ? (create a “jigsaw” org) 
xquad_r -> google
wikitext-> metamind (or salesforce)

// deprecate c4 and mc4 in favor of allenai/c4 (add a dataset script to the org version to make it easier to use?)
```
</details>

Also, a space that allows users to claim the existing canonical datasets (for themselves or their organizations) could be nice.

WDYT?

Next week I can take care of some of them :) In most cases we just need to send an email to ask them if they're ok with it.
Let's coordinate on slack ?

Yup, sounds good to me!

I can also continuing working on this if we agree this has become a priority now.

cool stuff! 

this morning on my side i  moved huggingface.co/ctrl (a not very used model) to its rightful entity

As a previous step before transferring the datasets, we decided we should convert them to Parquet, so that the viewer does not stop working (the viewer does not support datasets with scripts). 

Datasets converted to Parquet:
- [x] adversarial_qa
- [x] ai2_arc
- [x] americas_nli
- [x] anli
- [x] asset
- [x] beans
- [ ] bigbench
- [x] billsum
- [ ] blbooks: it was already transferred to: TheBritishLibrary/blbooks
- [ ] blbooksgenre: it was already transferred to: TheBritishLibrary/blbooksgenre
- [x] blimp
- [x] boolq
- [ ] c4
- [x] capes
- [ ] cc100
- [x] cc_news
- [x] cifar10
- [x] cifar100
- [x] civil_comments
- [x] clue
- [x] common_gen
- [x] commonsense_qa
- [ ] competition_math: it was already transferred to: hendrycks/competition_math
- [x] coqa
- [x] cos_e
- [ ] covost2: it requires manual download
- [x] cppe-5
- [x] dbpedia_14
- [x] discofuse
- [x] drop
- [x] duorc
- [x] exams
- [x] food101
- [x] glue
- [x] go_emotions
- [x] greek_legal_code
- [x] gsm8k
- [x] hate_speech_offensive
- [x] imdb
- [x] imppres
- [x] indic_glue
- [x] jfleg
- [x] kilt_tasks
- [x] klue
- [x] lambada
- [x] lex_glue
- [ ] librispeech_asr
- [x] mbpp
- [ ] mc4
- [x] medical_questions_pairs
- [x] medmcqa
- [x] ms_marco
- [ ] multi_eurlex
- [x] multi_nli
- [ ] narrativeqa
- [ ] news_commentary
- [x] nq_open
- [x] openai_humaneval
- [x] openbookqa
- [ ] opus_euconst
- [x] paws
- [x] paws-x
- [x] pubmed_qa
- [x] qasc
- [x] quail
- [x] quartz
- [ ] quickdraw
- [x] race
- [x] ropes
- [x] sciq
- [x] scitail
- [ ] snli
- [x] spider
- [x] squad
- [x] squad_v2
- [x] sst2
- [x] stsb_multi_mt
- [x] tiny_shakespeare
- [x] trivia_qa
- [x] truthful_qa
- [x] tweet_eval
- [x] web_questions
- [ ] wiki40b
- [x] wiki_lingua
- [x] wiki_qa
- [ ] wikiann
- [x] wikitext
- [x] wino_bias
- [x] xcopa
- [x] xcsr
- [x] xnli
- [x] xquad
- [x] xquad_r
- [ ] xtreme
- [x] yelp_review_full


For `c4` and `mc4` I was thinking of adding the corresponding configs to `allenai/c4` and redirect `c4` and `mc4` to `allenai/c4`. I'll open a PR on `allenai/c4` if it's good for you

@davanstrien and @lhoestq, I have shared with you this spreadsheet: https://docs.google.com/spreadsheets/d/1GvNTd1UxmtTvEFOK-Eq6E3Str4FUWQuWZsEN0WVFirs/edit?usp=sharing

This way we can take datasets by batches to contact the authors and transfer to the organizations.

We have already transferred all canonical datasets under organization/user namespaces.