pkg_resources' working_set caching breaks transformers import on google colab
Author: konstin
Pull request: no

## Environment info

- `transformers` version: 4.4.2
- Platform: google colab
- Python version: 3.7.10
- PyTorch version (GPU?): n/a
- Tensorflow version (GPU?):  n/a
- Using GPU in script?:  n/a
- Using distributed or parallel set-up in script?:  n/a

### Who can help

CC @stas00 as this was implemented in #8645

## To reproduce

You can find a complete example in [this google colab](https://colab.research.google.com/drive/1WT7GSd4uk9TLle9q9ftRNFy4BLZWCvZa?usp=sharing), also exported to [this gist](https://gist.github.com/konstin/e42d8f428fa11ba389e31be69cdc5646).

To reproduce, first [create a new google colab notebook](https://colab.research.google.com/#create=true). Let's install recent transformers and tqdm versions in it:

```
!pip install -U pip
!pip install -U "transformers<5.0.0,>=4.0.0" "tqdm<5.0.0,>=4.45.0"
```

This currently install transformers 4.4.2 and tqdm 4.59.0.

Surprisingly, now running `import transformers` fails. We get an error in pkg_resources, which is looking for the .dist-info of tqdm 4.41.1, when the installed version is 4.59.0:

```
[...]
/usr/local/lib/python3.7/dist-packages/pkg_resources/__init__.py in _get(self, path)
   1609 
   1610     def _get(self, path):
-> 1611         with open(path, 'rb') as stream:
   1612             return stream.read()
   1613 

FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.7/dist-packages/tqdm-4.41.1.dist-info/METADATA'
```

The cause is that pkg_resources uses the cached [WorkingSet](https://setuptools.readthedocs.io/en/latest/pkg_resources.html#workingset-objects), which contains the state before the pip install. We can confirm this by recreating pkg_resources' cache manually:

```python
import pkg_resources

pkg_resources.working_set = pkg_resources.WorkingSet()
```

Afterwards, importing transformers works.

The above example is the minimized version of our real [notebooks examples](https://github.com/sacdallago/bio_embeddings/tree/develop/notebooks):

```python
!pip install -U pip
!pip install -U bio_embeddings[all]

from bio_embeddings.embed import SeqVecEmbedder # This line fails with the tqdm .dist-info not found error
```

## Expected behavior

transformers should use the actual installed versions for checking compatibility instead of pkg_resources cache. This could be achieved e.g. by using [importlib_metadata](https://github.com/python/importlib_metadata) instead of pkg_resources [here](https://github.com/huggingface/transformers/blob/1c06240e1b3477728129bb58e7b6c7734bb5074e/src/transformers/utils/versions.py#L80) or by recreating pkg_resources` cache with `pkg_resources.working_set = pkg_resources.WorkingSet()` before checking versions.

I've used the following snippet to check that importlib_metadata works, which prints `4.41.1` and `4.59.0`:

```python
import pkg_resources
import importlib_metadata

print(pkg_resources.get_distribution("tqdm").version)
print(importlib_metadata.version("tqdm"))
```

I can prepare a pull request for either solution.

Comments:
Thank you for this awesome report, @konstin, and identifying the cause of the problem and the solution.

This looks like a bug in `pkg_resources`. It should update its cache after install. 

The other workaround that seems to work is to just do:
```
!pip install -U "transformers<5.0.0,>=4.0.0" "tqdm<5.0.0,>=4.45.0"
!pip install -U "transformers<5.0.0,>=4.0.0" "tqdm<5.0.0,>=4.45.0"
```
So the 2nd one updates the cache.

You may want to report this bug to `pkg_resources`.

Since `transformers` has started using `importlib_metadata` extensively as of recent I think your proposed solution sounds great, so yes please - the proposed PR sounds perfect.

Thank you.