'SummaryWriter' object has no attribute 'add_hparams'
Author: ZiningZhu
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.4.0
- Platform: Linux-4.15.0-72-generic-x86_64-with-debian-buster-sid
- Python version: 3.6.12
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Tried both 1 gpu and 2 gpus. Got the same result.

Additional env information from `pip freeze`:  
- tensorboardX==1.6  
- tensorflow==2.2.0 (I did not include tensorflow in this current conda environment, but do have that in the system, so I think pip reads from that. `import tensorflow` in a python script would cause `ImportError`, so tensorflow should be considered uninstalled here).  

### Who can help
@sgugger 

## Information

Model I am using (Bert, XLNet ...): `bert-base-cased`

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below; in steps to reproduce the situation)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name) MNLI
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Copy the `run_glue.py` from [cdc48ce](https://github.com/huggingface/transformers/commit/cdc48ce92ddf50e7ad871376be651638268b2e9a) (the newest version up till now).  
2. Comment out the `from transformers.trainer_utils import is_main_process` line, and insert below (because this importing throws some exception. Pasting this code circumvents the problem):  
```

def is_main_process(local_rank):
    """
    Whether or not the current process is the local process,basedon`local_rank`.
    """
    return local_rank in [-1, 0]
```
3. Run the following scripts.
```
export GLUE_DIR=../../data/glue_data
export TASK_NAME=MNLI
python run_glue.py \
  --model_name_or_path bert-base-cased \
  --task_name $TASK_NAME \
  --do_train \
  --do_eval \
  --do_predict \
  --max_seq_length 128 \
  --per_device_train_batch_size 8 \
  --learning_rate 2e-5 \
  --num_train_epochs 2 \
  --output_dir $TASK_NAME/
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

The error message is:
```
Traceback (most recent call last):
  File "run_glue.py", line 421, in <module>
    main()
  File "run_glue.py", line 356, in main
    model_path=model_args.model_name_or_path if os.path.isdir(model_args.model_name_or_path) else None
  File "/h/zining/.conda/envs/myenv/lib/python3.6/site-packages/transformers/trainer.py", line 717, in train
    self.control = self.callback_handler.on_train_begin(self.args, self.state, self.control)
  File "/h/zining/.conda/envs/myenv/lib/python3.6/site-packages/transformers/trainer_callback.py", line 329, in on_train_begin
    return self.call_event("on_train_begin", args, state, control)
  File "/h/zining/.conda/envs/myenv/lib/python3.6/site-packages/transformers/trainer_callback.py", line 376, in call_event
    **kwargs,
  File "/h/zining/.conda/envs/myenv/lib/python3.6/site-packages/transformers/integrations.py", line 218, in on_train_begin
    self.tb_writer.add_hparams(args.to_sanitized_dict(), metric_dict={})
AttributeError: 'SummaryWriter' object has no attribute 'add_hparams'
```


## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
I think running the `run_glue.py` will finetune on some GLUE tasks.  

Note: Issue #4511 is similar, but was threw in `trainer.py`. My issue is thrown in `trainer_callback.py`. I think these two issues are caused by different reasons.  

Comments: