Refac module factory + avoid etag requests for hub datasets
Opened by lhoestq (closed)

## Refactor the module factory

When trying to extend the `data_files` logic to avoid doing unnecessary ETag requests, I noticed that the module preparation mechanism needed a refactor:
- the function was 600 lines long
- it was not readable
- it contained many different cases that made it complex to maintain
- it was hard to properly test it
- it was hard to extend without breaking anything

The module preparation mechanism is in charge of taking the name of a dataset or a metric given by the user (ex: "squad", "accuracy", "lhoestq/test", "path/to/my/script.py", "path/to/my/data/directory", "json", "csv") and return a module (possibly downloaded from the Hub) that contains the dataset builder or the metric class to use.

### Implementation details

I decided to separate all these use cases into different dataset/metric module factories.

First, the metric module factories:
- **CanonicalMetricModuleFactory**: "accuracy", "rouge", ...
- **LocalMetricModuleFactory**: "path/to/my/metric.py"

Then, the dataset module factories:
- **CanonicalDatasetModuleFactory**: "squad", "glue", ...
- **CommunityDatasetModuleFactoryWithScript**: "lhoestq/test"
- **CommunityDatasetModuleFactoryWithoutScript**: "lhoestq/demo1"
- **PackagedDatasetModuleFactory**: "json", "csv", ...
- **LocalDatasetModuleFactoryWithScript**: "path/to/my/script.py"
- **LocalDatasetModuleFactoryWithoutScript**: "path/to/my/data/directory"

And finally, additional factories when users have no internet:
- **CachedDatasetModuleFactory**
- **CachedMetricModuleFactory**

### Breaking changes

One thing is that I still don't know at what extent we want to keep backward compatibility for `prepare_module`. For now I just kept it (except I removed two parameters) just in case, but it's not used anywhere anymore.

## Avoid etag requests for hub datasets

To do this I added a class `DataFilesDict` that can be hashed to define the cache directory  of the dataset.
It contains the usual data files formatted as `{"train": ["train.txt"]}` for example.
But each list of file is a `DataFilesList` that also has a `origin_metadata` attribute that contains metadata about the origin of each file:
- for URLs: it stores the ETags of the files
- for local files: it stores the last modification data
- for files from a Hugging Face repository on the Hub: it stores the pattern (`*`, `*.csv`, "train.txt", etc.) and the commit sha of the repository (so there're no ETag requests !)

This way if any file changes, the hash of the `DataFilesDict` changes too !

You can instantiate a `DataFilesDict` by using patterns for local/remote files or files in a HF repository:
- for local/remote files: `DataFilesDict.from_local_or_remote(patterns)`
- for files in a HF repository: `DataFilesDict.from_hf_repo(patterns, dataset_info)`

Fix #2859 

## TODO

Fix the latest test:
- [x] fix the call to dataset_info in offline mode (related to https://github.com/huggingface/huggingface_hub/issues/372)

Add some more tests:
- [x] test all the factories
- [x] test the new data files logic

Other:
- [x] docstrings
- [x] comments

Comments:
> One thing is that I still don't know at what extent we want to keep backward compatibility for prepare_module. For now I just kept it (except I removed two parameters) just in case, but it's not used anywhere anymore.

FYI, various other projects currently use it, thus clearly a major version would be required:

https://github.com/search?q=org%3Ahuggingface+prepare_module&type=code

Yea so I kept `prepare_module` and changed it to use all the factories I added, so all the use cases in the link you shared are still working. The only two parameters I removed are minor IMO and were a bit hacky anyway (return_resolved_file_path and return_associated_base_path). I think they were only used internally in `datasets` but let me know if you're aware of a use case I didn't think of.

I think I'm done with the tests :) I'll do the comments/docs and then we just wait for https://github.com/huggingface/huggingface_hub/pull/373 to get merged

When there's a new release of `huggingface_hub` (probably on monday), it will fix the CI.

The PR is ready for review. Let me know if I need to clarify some parts

One additional change I did: the tests won't affect the number of downloads on the website anymore. And users can choose to not update the number of downloads with `HF_UPDATE_DOWNLOAD_COUNTS=0`

CI failures are simply due to RAM issues with circleci workers.
And on windows there is an issue with installing `ruamel.yaml` from the bump of `huggingface_hub` (fixed on master)