Discuss whether support canonical datasets w/o dataset_infos.json and/or dummy data
IMO, the data streaming test is good enough of a test that the dataset works correctly (assuming that we can more or less ensure that if streaming works then the non-streaming case will also work), so that for datasets that have a working dataset preview, we can remove the dummy data IMO. On the other hand, it seems like not all datasets have streaming enabled yet and for those datasets (if they are used a lot), I think it would be nice to continue testing some dummy data.

I don't really have an opinion regarding the JSON metadata as I don't know enough about it.

 I don't know all the details, but generally I'd be in favor of unifying the metadata formats into YAML inside .md (and so deprecating the dataset_infos.json) 

(Ultimately the CI can run on "HuggingFace Actions" instead of on GitHub) The dataset_infos.json file currently has these useful infos for each dataset configuration, that I think can be moved to the dataset tags:
- Size of the dataset in MB: download size, arrow file size, and total size (sum of download + arrow)
- Size of each split in MB and number of examples. Again this can be moved to the dataset tags
- Feature type of each column
- supported task templates (it defines what columns correspond to the features and labels for example)

But it also has this, which I'm not sure if it should be in the tags or not:
- Checksums of the downloaded files for integrity verifications

So ultimately this file could probably be deprecated in favor of having the infos in the tags.

> Also note that for generating both (dataset_infos.json file and dummy data), the entire dataset needs being downloaded. This can be an issue for huge datasets (like WIT, with 400 GB of data).

To get the exact number of examples and size in MB of the dataset, one needs to download and generate it completely. IMO these infos are very important when someone considers using a dataset. Though using streaming we could do some extrapolation to have approximate values instead.

For the integrity verifications we also need the number of examples and the checksums of the downloaded files, so it requires the dataset to be fully downloaded once. This can be optional though.

> IMO, the data streaming test is good enough of a test that the dataset works correctly (assuming that we can more or less ensure that if streaming works then the non-streaming case will also work)

I agree with this. Usually if a dataset works in streaming mode, then it works in non-streaming mode (the other way around is not true though).

> On the other hand, it seems like not all datasets have streaming enabled yet and for those datasets (if they are used a lot), I think it would be nice to continue testing some dummy data.

Yes indeed, or at least make sure that it was tested on the true data. (note that if we wanted to display sizes, etc we could also pretty easily parse the `dataset_infos.json` on the hub side) I agree that we can move the relevant parts of `dataset_infos.json` to the YAML tags.

> On the other hand, it seems like not all datasets have streaming enabled yet and for those datasets (if they are used a lot), I think it would be nice to continue testing some dummy data. <
> > Yes indeed, or at least make sure that it was tested on the true data.

I like the idea of testing streaming and falling back to the dummy data test if streaming does not work. Generating dummy data can be very tedious, so this would be a nice incentive for the contributors to make their datasets streamable.  
I open this PR to have a public discussion about this topic and make a decision.

As previously discussed, once we have the metadata in the dataset card (README file, containing both Markdown info and YAML tags), what is the point of having also the JSON metadata (dataset_infos.json file)?

On the other hand, the dummy data is necessary for testing (in our CI suite) that the canonical dataset loads correctly. However:
- the dataset preview feature is already an indirect test that the dataset loads correctly (it also tests it is streamable though)
- we are migrating canonical datasets to the Hub
Do we really need to continue testing them in out CI?

Also note that for generating both (dataset_infos.json file and dummy data), the entire dataset needs being downloaded. This can be an issue for huge datasets (like WIT, with 400 GB of data).

Feel free to ping other people for the discussion.

CC: @lhoestq @mariosasko @thomwolf @julien-c @patrickvonplaten @anton-l @LysandreJik @yjernite @nateraw 