Load GitHub datasets from Hub
Opened by albertvillanova (closed)

We have recurrently had connection errors when requesting GitHub because sometimes the site is not available.

This PR requests the Hub instead, once all GitHub datasets are mirrored on the Hub.

Fix #2048

Related to:
- #4051 
- #3210
- #2787 
- #2075
- #2036

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Currently the github datasets versioning is synced with the `datasets` lib versioning: when you load a github dataset using `datasets==x.y.z`, then the version of the dataset will be the one at the git tag `x.y.z`. This is for reproducibility reasons.

We could stop having this behavior and always use the latest version of the dataset, but when we do a breaking change it will break github datasets for previous versions of the library. It could be nice to think about tools that will allow backward compatibility if we ever need to to a breaking change in some datasets. Maybe a way to specify which revision of the dataset to use based on the `datasets` major version.

If we keep this behavior, then maybe add a note in setup.py to push to PyPI only after the `Update Hub repositories` CI job is done. It can take a few minutes to add the version tag to all the dataset repositories on the Hub. If we push to PyPI before the tags are pushed, then some users might get some 404 if at the same time they installed `datasets` and run `load_dataset`.

@lhoestq I was going to increase the `max_retries` as done for metrics:
- #4063 

But then I realized that loading from the Hub would work as well. That is why I opened this PR.

Definitely, we should decide which behavior we want:
- We have been working in the direction of eliminating the distinctions between canonical/community datasets
- If we continue to go in that direction, then passing (or not passing) `revision` should have the same behavior for canonical/community
- If we want to continue to tight the library version with the canonical datasets version, that is definitely a difference between canonical and community datasets

Not sure what could be better in the long term...

> We could stop having this behavior and always use the latest version of the dataset, but when we do a breaking change it will break github datasets for previous versions of the library. 

Not sure of understanding this. Previous versions of the `datasets` library will continue to download GitHub datasets from GitHub, syncing library/dataset versions... Where is the problem?

Yes you're right, previous versions of `datasets` will still continue to download from github, but not future versions.
If we release `datasets` 2.1 by removing this behavior and if one day we release `datasets` 3.0 with a breaking change in the dataset scripts, then all version >=2.1 will break.

Ideally we should drop the differences between github datasets and community datasets, and maybe provide a way to fallback on an older version of a dataset repository if the user's `datasets` version is too old and incompatible with it.

I just noticed I literally opened the same PR lol

I'm still convinced that we should do a better version compatibility check but we can see that later IMO

Normally in open source projects, when there is a duplicate PR, the latter is tagged as "duplicate" and closed. :stuck_out_tongue_winking_eye: 

Let me make things clear in my mind: so you say that the blocking point that was preventing this PR from merging, now is no longer a blocking point and could be addresses in a subsequent PR?

Let me close the duplicate one, sorry

> Let me make things clear my mind: so you say that the blocking point that was preventing this PR from merging now is no longer a blocking point and could be addresses in a subsequent PR?

Yes 🙈

> Note that after this PR, all the changes made to a dataset will affect all the datasets version from now on

Yes, we have aligned this behavior with Hub datasets, as this is already the case for Hub datasets.