Add local_files_only parameter to pretrained items
Author: BramVanroy
Pull request: yes

closes https://github.com/huggingface/transformers/issues/2867

Setting local_files_only=True disables outgoing traffic:
- etags are not looked up
- files are not downloaded (config, tokenizer, model)

An appropriate error is thrown when this argument may be the cause why a model cannot be loaded.

```python
import pyinstrument
from transformers import DistilBertConfig, DistilBertModel, DistilBertTokenizer


class TreeProfiler():
    def __init__(self, show_all=False):
        self.profiler = pyinstrument.Profiler()
        self.show_all = show_all # verbose output of pyinstrument profiler

    def __enter__(self):
        print("WITH TREE_PROFILER:")
        self.profiler.start()

    def __exit__(self, *args):
        self.profiler.stop()
        print(self.profiler.output_text(unicode=True, color=True, show_all=self.show_all))


def main():
    with TreeProfiler(show_all=True):
        config = DistilBertConfig.from_pretrained('distilbert-base-uncased', local_files_only=True)
        model = DistilBertModel.from_pretrained('distilbert-base-uncased', local_files_only=True)
        tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased', local_files_only=True)


if __name__ == '__main__':
    main()
```

The above snippet will throw an error message when the expected files are not present in the cache.  When they are, though, everything is loaded fine without the need of any additional lookups.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2930?src=pr&el=h1) Report
> Merging [#2930](https://codecov.io/gh/huggingface/transformers/pull/2930?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/59c23ad9c931ac4fe719abeb3c3851df046ef3a6?src=pr&el=desc) will **decrease** coverage by `1.05%`.
> The diff coverage is `92.3%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2930/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2930?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #2930      +/-   ##
==========================================
- Coverage    75.3%   74.24%   -1.06%     
==========================================
  Files          94       94              
  Lines       15424    15430       +6     
==========================================
- Hits        11615    11456     -159     
- Misses       3809     3974     +165
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2930?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/configuration\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2930/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX3V0aWxzLnB5) | `96.49% <100%> (+0.03%)` | :arrow_up: |
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2930/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `90.1% <100%> (+0.01%)` | :arrow_up: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2930/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `61.46% <100%> (+0.13%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2930/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `68% <88.88%> (+0.25%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_pytorch\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2930/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9weXRvcmNoX3V0aWxzLnB5) | `8.72% <0%> (-81.21%)` | :arrow_down: |
| [src/transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/2930/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19yb2JlcnRhLnB5) | `85.91% <0%> (-9.86%)` | :arrow_down: |
| [src/transformers/modeling\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/2930/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ194bG5ldC5weQ==) | `70.86% <0%> (-2.3%)` | :arrow_down: |
| [src/transformers/modeling\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/2930/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19jdHJsLnB5) | `92.07% <0%> (-2.21%)` | :arrow_down: |
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/2930/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `80.2% <0%> (-1.35%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2930?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2930?src=pr&el=footer). Last update [59c23ad...826eced](https://codecov.io/gh/huggingface/transformers/pull/2930?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


I think this feature is reasonable but I'm not sure about the param name. Maybe something like `disable_networking` or `local_files_only`?

Yeah, wasn't sure about a parameter name. I like `local_files_only`, even though it's quite long.

Would it make sense to utilise this in the examples? I am thinking about multi-GPU set-ups where the online lookup only has to be done by the main process (local_rank == 0). For all other processes, local_files_only can be True. Might avoid some redundant look-ups - even though in practice it won't matter much in terms of speed (couple of seconds at most).