Transformers 4.1.1 & Tensorflow 2.0, AttributeError: module'tensorflow_core.keras.activations' has no attribute'swish' 
Author: kangzhiheng
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.1.1
- Platform: Mac
- Python version: 3.6
- PyTorch version (GPU?): No
- Tensorflow version (GPU?): 2.0.0 No GPU
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

## Information
@Rocketknight1 @sgugger

I had to use `tensorflow2.0` for some reasons. I checked all transfomer released versions and found that `before 4.1.1`, tensorflow>=2.0, after 4.1.1, tensorflow>=2.3 (in `setup.py`), so I `4.1.1` is installed.
When I run 
```
from transformers import AutoTokenizer, AutoModel code
```
I raise 
```
AttributeError: module'tensorflow_core.keras.activations' has no attribute'swish' 
```
So I checked the `keras` document of tf2.0 (https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/activations), `there is indeed no swish function`, but it does exist in the `"transformers\activations_tf.py"`(line 64~74) in transformers `4.1.1`:
```
ACT2FN = {
    "gelu": tf.keras.layers.Activation(gelu),
    "relu": tf.keras.activations.relu,
    "swish": tf.keras.activations.swish,
    "silu": tf.keras.activations.swish,
    "gelu_new": tf.keras.layers.Activation(gelu_new),
    "mish": tf.keras.layers.Activation(mish),
    "tanh": tf.keras.activations.tanh,
    "gelu_fast": tf.keras.layers.Activation(gelu_fast),
}
```

## Expected behavior
When I modify `activations_tf.py`, it seems like ok...
```
ACT2FN = {
    "gelu": tf.keras.layers.Activation(gelu),
    "relu": tf.keras.activations.relu,
    # "swish": tf.keras.activations.swish,
    # "silu": tf.keras.activations.swish,
    "gelu_new": tf.keras.layers.Activation(gelu_new),
    "mish": tf.keras.layers.Activation(mish),
    "tanh": tf.keras.activations.tanh,
    "gelu_fast": tf.keras.layers.Activation(gelu_fast),
}
```

or 

define `swish` and `silu` in `activations_tf.py` like

```
def swish():
     xxxx

def silu():
    xxxx

ACT2FN = {
    "gelu": tf.keras.layers.Activation(gelu),
    "relu": tf.keras.activations.relu,
    "swish": tf.keras.layers.Activation(swish),
    "silu": tf.keras.layers.Activation(swish),
    "gelu_new": tf.keras.layers.Activation(gelu_new),
    "mish": tf.keras.layers.Activation(mish),
    "tanh": tf.keras.activations.tanh,
    "gelu_fast": tf.keras.layers.Activation(gelu_fast),
}
```

I don’t know if there is such a bug.  @Rocketknight1 @sgugger.

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
The setup was updated super late: we already required tensorflow >= 2.3 for a while when we finally went to it. I don't know which version of Transformers supports tensorflow 2.0 but I would guess it's 3.0 or even below.

Okay, hope the document can be clearer. Every time I read `README.md`, it says `This repository is tested on Python 3.6+, PyTorch 1.0.0+ (PyTorch 1.3.1+ for examples) and TensorFlow 2.0` similar words, I really think it is the specific version 2.0 of tensorflow. However, it is not(tensorflow>=2.3 in `setup.py`). It would be better if the version information is clearer I guess.
Hope `transformers` get better and better.

Yes, this part of the README has not been updated in a while (the PyTorch version is also wrong). Will adjust!

> Yes, this part of the README has not been updated in a while (the PyTorch version is also wrong). Will adjust!

Thanks 🤗.