Support running tests in parallel
Author: aaugustin
Pull request: yes

At this point, after `pip install pytest-xdist`, I'm getting a **2.5x speedup** running tests locally on my 2016 MBP (2,9 GHz Quad-Core Intel Core i7):

- `python -m pytest -n auto -s -v ./transformers/tests/` runs in slightly less than 2 minutes
- `python -m pytest -s -v ./transformers/tests/` takes slightly more than 5 minutes

Furthermore, Circle CI gets a **2,15x speedup**, going from [7:30 minutes](https://circleci.com/workflow-run/4ca98875-2d5c-438b-951f-4939d2f3cfc9) to [3:30 minutes](https://circleci.com/workflow-run/ad7094bb-0a0b-404d-ba3f-bd18b37f98bd).

The bottleneck is now the examples, which take a bit less than 3:30 to run, even with parallelization.

This PR adds a new dependency: filelock. You'll need to `pip install -e .` for local development again after it's merged.

This is now ready for review.

----

EDIT - test run time jumped up after I rebased on top of master mostly because of #2246.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2217?src=pr&el=h1) Report
> Merging [#2217](https://codecov.io/gh/huggingface/transformers/pull/2217?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/ac1b449cc938bb34bc9021feff599cfd3b2376ae?src=pr&el=desc) will **increase** coverage by `0.15%`.
> The diff coverage is `59.43%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2217/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2217?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #2217      +/-   ##
==========================================
+ Coverage   79.82%   79.97%   +0.15%     
==========================================
  Files         131      131              
  Lines       19496    19427      -69     
==========================================
- Hits        15562    15537      -25     
+ Misses       3934     3890      -44
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2217?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [transformers/tests/utils.py](https://codecov.io/gh/huggingface/transformers/pull/2217/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL3V0aWxzLnB5) | `94.11% <100%> (+0.36%)` | :arrow_up: |
| [transformers/tests/modeling\_distilbert\_test.py](https://codecov.io/gh/huggingface/transformers/pull/2217/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX2Rpc3RpbGJlcnRfdGVzdC5weQ==) | `99.21% <100%> (ø)` | :arrow_up: |
| [transformers/tests/modeling\_tf\_distilbert\_test.py](https://codecov.io/gh/huggingface/transformers/pull/2217/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX3RmX2Rpc3RpbGJlcnRfdGVzdC5weQ==) | `99.08% <100%> (ø)` | :arrow_up: |
| [transformers/tests/modeling\_tf\_auto\_test.py](https://codecov.io/gh/huggingface/transformers/pull/2217/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX3RmX2F1dG9fdGVzdC5weQ==) | `40.67% <11.11%> (ø)` | :arrow_up: |
| [transformers/tests/modeling\_tf\_albert\_test.py](https://codecov.io/gh/huggingface/transformers/pull/2217/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX3RmX2FsYmVydF90ZXN0LnB5) | `96.19% <33.33%> (+1.74%)` | :arrow_up: |
| [transformers/tests/modeling\_tf\_transfo\_xl\_test.py](https://codecov.io/gh/huggingface/transformers/pull/2217/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX3RmX3RyYW5zZm9feGxfdGVzdC5weQ==) | `95.87% <50%> (+1.87%)` | :arrow_up: |
| [transformers/tests/modeling\_tf\_openai\_gpt\_test.py](https://codecov.io/gh/huggingface/transformers/pull/2217/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX3RmX29wZW5haV9ncHRfdGVzdC5weQ==) | `96.39% <50%> (+1.65%)` | :arrow_up: |
| [transformers/tests/modeling\_transfo\_xl\_test.py](https://codecov.io/gh/huggingface/transformers/pull/2217/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX3RyYW5zZm9feGxfdGVzdC5weQ==) | `96.26% <50%> (+1.71%)` | :arrow_up: |
| [transformers/tests/modeling\_xlm\_test.py](https://codecov.io/gh/huggingface/transformers/pull/2217/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX3hsbV90ZXN0LnB5) | `97.36% <50%> (+1.23%)` | :arrow_up: |
| [transformers/tests/modeling\_roberta\_test.py](https://codecov.io/gh/huggingface/transformers/pull/2217/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX3JvYmVydGFfdGVzdC5weQ==) | `76.15% <50%> (+0.96%)` | :arrow_up: |
| ... and [16 more](https://codecov.io/gh/huggingface/transformers/pull/2217/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2217?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2217?src=pr&el=footer). Last update [ac1b449...b8e924e](https://codecov.io/gh/huggingface/transformers/pull/2217?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


I'm getting an unexpected result on Circle CI:

- TensorFlow tests are about 4 times faster, which makes sense
- PyTorch tests are about 4 times **slower?!**

I'm talking about the time spent in pytest here, which is shown at the bottom of each test run.

**Before parallelization**

https://circleci.com/workflow-run/4ca98875-2d5c-438b-951f-4939d2f3cfc9

04:08 build_py3_tf
00:59 build_py3_torch

05:21 build_py3_torch_and_tf

04:30 build_py2_tf
01:12 build_py2_torch

**After parallelization**

https://circleci.com/workflow-run/5d273bdb-0b4d-4e71-b0f1-272d3f9f72da

00:56 build_py3_tf
04:02 build_py3_torch

04:53 build_py3_torch_and_tf

01:25 build_py2_tf
05:39 build_py2_torch

----

EDIT - I thought this might happen if the PyTorch tests do expensive calculations in setUp / setUpClass. Due to how pytest-xdist works, setUp / setUpClass may run multiple times on different CPUs. However, the `--dist=loadfile` option of pytest-xdist doesn't seem to help, so it must be something else.

----

EDIT 2 - setting OMP_NUM_THREADS=1 solves this, thanks @mfuntowicz!

I'm also facing an issue with `hf_api_test.py`. I'm hitting a HTTP 401 error when running tests in parallel, and so does Circle CI. Are we blocked by a security rule protecting against excessive login requests?

I'm not a big fan of tests that depend on network services. Often they're flaky.

EDIT -- this is solved by the `--dist=loadfile` option of pytest-xdist.

For future reference, I logged all filesystem read/writes with:

```
sudo opensnoop -F -n python \
    | grep -v "$HOME/\.pyenv" \
    | grep -v "$HOME/\.virtualenvs" \
    | grep -v "__pycache__" \
    | grep -v "$TMPDIR" \
    | grep -v "0x00000000 \." \
    > open.log
```

while running the full test suite:

```
RUN_SLOW=1 python -m unittest discover -s transformers/tests -p '*_test.py' -t . -v
```

This doesn't reveal anything that could collide between tests.