tensorflow has no attribute swish
Author: FrancoisMentec
Pull request: no

## Environment info
- transformers version: 4.6.1
- Platform: Linux-5.4.0-1047-azure-x86_64-with-glibc2.10
- Python version: 3.8.1
- PyTorch version (GPU?): 1.8.1 (True)
- Tensorflow version (GPU?): 2.5.0 (False)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

```
azureuser@ai4hr-k80:~/cloudfiles/code/Users/francois.mentec$ pip show tensorflow
Name: tensorflow
Version: 2.5.0
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: /anaconda/envs/azureml_py38/lib/python3.8/site-packages
Requires: typing-extensions, h5py, grpcio, tensorflow-estimator, wrapt, gast, tensorboard, six, keras-nightly, astunparse, flatbuffers, wheel, absl-py, protobuf, numpy, opt-einsum, keras-preprocessing, termcolor, google-pasta
Required-by: 
azureuser@ai4hr-k80:~/cloudfiles/code/Users/francois.mentec$ pip show transformers
Name: transformers
Version: 4.6.1
Summary: State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch
Home-page: https://github.com/huggingface/transformers
Author: Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Sam Shleifer, Patrick von Platen, Sylvain Gugger, Suraj Patil, Stas Bekman, Google AI Language Team Authors, Open AI team Authors, Facebook AI Authors, Carnegie Mellon University Authors
Author-email: thomas@huggingface.co
License: Apache
Location: /anaconda/envs/azureml_py38/lib/python3.8/site-packages
Requires: numpy, filelock, requests, tqdm, sacremoses, packaging, tokenizers, huggingface-hub, regex
Required-by: 
```

### Who can help
- tensorflow: @Rocketknight1

## To reproduce

Steps to reproduce the behavior:
import transformers:
`from transformers import BertTokenizer, BertModel, AdamW`

## Expected behavior

Following error:
`AttributeError: module 'tensorflow_core.python.keras.api._v2.keras.activations' has no attribute 'swish'`

Comments:
This is very strange - I can't reproduce it here, and Tensorflow >= 2.3 should have Swish as an activation. I'll try to figure that one out, but if you discover anything else about the problem, let me know!

Tensorflow version is wrong:
```
import tensorflow as ts

print(ts.__version__)
```
give:
`2.1.0`

I have no idea why pip and transformers-cli show a different version. I must state that I hate Microsoft Azure. The error isn't on your side.

EDIT:
For anyone who get the same error with Azure, you need to select Python 3.8 in the kernel version on your notebook, Python 3 is selected by default.
![image](https://user-images.githubusercontent.com/22057576/120608179-32496d00-c451-11eb-99ad-72c06e0bbb7f.png)


Don't worry about it! This is something I see with conda or venvs sometimes - usually it's because I'm accidentally using two Python environments at once - for example, if I don't have pip installed in a conda environment the pip command still works, but it's actually the system pip, and if I'm not paying attention I just end up installing packages systemwide, while still being unable to access them in my active environment.