GitHub issue #2986: Refac module factory + avoid etag requests for hub datasets
State: closed | Created: 2021-09-29T10:42:00Z

## Refactor the module factory

When trying to extend the `data_files` logic to avoid doing unnecessary ETag requests, I noticed that the module preparation mechanism needed a refactor:
- the function was 600 lines long
- it was not readable
- it contained many different cases that made it complex to maintain
- it was hard to properly test it
- it was hard to extend without breaking anything

The module preparation mechanism is in charge of taking the name of a dataset or a metric given by the user (ex: "squad", "accuracy", "lhoestq/test", "path/to/my/script.py", "path/to/my/data/directory", "json", "csv") and return a module (possibly downloaded from the Hub) that contains the dataset builder or the metric class to use.

### Implementation details

I decided to separate all these use cases into different dataset/metric module factories.

First, the metric module factories:
- **CanonicalMetricModuleFactory**: "accuracy", "rouge", ...
- **LocalMetricModuleFactory**: "path/to/my/metric.py"

Then, the dataset module factories:
- **CanonicalDatasetModuleFactory**: "squad", "glue", ...
- **CommunityDatasetModuleFactoryWithScript**: "lhoestq/test"
- **CommunityDatasetModuleFactoryWithoutScript**: "lhoestq/demo1"
- **PackagedDatasetModuleFactory**: "json", "csv", ...
- **LocalDatasetModuleFactoryWithScript**: "path/to/my/script.py"
- **LocalDatasetModuleFactoryWithoutScript**: "path/to/my/data/directory"

And finally, additional factories when users have no internet:
- **CachedDatasetModuleFactory**
- **CachedMetricModuleFactory**

### Breaking changes

One thing is that I still don't know at what extent we want to keep backward compatibility for `prepare_module`. For now I just kept it (except I removed two parameters) just in case, but it's not used anywhere anymore.

## Avoid etag requests for hub datasets

To do this I added a class `DataFilesDict` that can be hashed to define the cache directory  of the dataset.
It contains the usual data files formatted as `{"train": ["train.txt"]}` for example.
But each list of file is a `DataFilesList` that also has a `origin_metadata` attribute that contains metadata about the origin of each file:
- for URLs: it stores the ETags of the files
- for local files: it stores the last modification data
- for files from a Hugging Face repository on the Hub: it stores the pattern (`*`, `*.csv`, "train.txt", etc.) and the commit sha of the repository (so there're no ETag requests !)

This way if any file changes, the hash of the `DataFilesDict` changes too !

You can instantiate a `DataFilesDict` by using patterns for local/remote files or files in a HF repository:
- for local/remote files: `DataFilesDict.from_local_or_remote(patterns)`
- for files in a HF repository: `DataFilesDict.from_hf_repo(patterns, dataset_info)`

Fix #2859 

## TODO

Fix the latest test:
- [x] fix the call to dataset_info in offline mode (related to https://github.com/huggingface/huggingface_hub/issues/372)

Add some more tests:
- [x] test all the factories
- [x] test the new data files logic

Other:
- [x] docstrings
- [x] comments