xla_spawn.py:  xm.get_ordinal() got an unexpected keyword argument 'local'
Author: BassaniRiccardo
Pull request: no

## Environment info

- `transformers` version: 4.7.0.dev0
- Platform: Linux-5.4.109+-x86_64-with-Ubuntu-18.04-bionic (working on Colab with TPU)
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.1+cu101 (False)
- Tensorflow version (GPU?): 2.5.0 (False)
- Using GPU in script?: No, using TPU
- Using distributed or parallel set-up in script?: number_of_cores = 8

### Who can help
<@sgugger, @patil-suraj -->

## Information

Model I am using: Bert

The problem arises when using:
* [ ] the official example scripts:
```ruby
python /transformers/examples/pytorch/xla_spawn.py --num_cores=8 \
           /transformers/examples/pytorch/language-modeling/run_mlm.py (--run_mlm.py args)
```

The tasks I am working on is:
* Pretraining BERT with TPU 

## To reproduce

Steps to reproduce the behavior:

1. install necessary packages:

```ruby
pip install git+https://github.com/huggingface/transformers
cd /content/transformers/examples/pytorch/language-modeling
pip install -r requirements.txt
pip install cloud-tpu-client==0.10 https://storage.googleapis.com/tpu-pytorch/wheels/torch_xla-1.8.1-cp37-cp37m-linux_x86_64.whl
```

2. run xla_spwan with minimal args passed to run_mlm: specify a small .txt TRAIN_FILE and an OUTPUT_DIR:

```ruby
python xla_spawn.py \
        --num_cores=8 \
        language-modeling/run_mlm.py \
            --train_file $TRAIN_FILE \
            --model_name_or_path bert-base-uncased \
            --output_dir $OUTPUT_DIR \
            --overwrite_output_dir True \
            --do_train True \
```

I get this error (for different TPU cores):

```ruby
Exception in device=TPU:0: get_ordinal() got an unexpected keyword argument 'local'
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 329, in _mp_start_fn
    _start_fn(index, pf_cfg, fn, args)
  File "/usr/local/lib/python3.7/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 323, in _start_fn
    fn(gindex, *args)
  File "/content/drive/My Drive/Thesis/transformers/examples/pytorch/language-modeling/run_mlm.py", line 493, in _mp_fn
    main()
  File "/content/drive/My Drive/Thesis/transformers/examples/pytorch/language-modeling/run_mlm.py", line 451, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/usr/local/lib/python3.7/dist-packages/transformers/trainer.py", line 1193, in train
    self.state.is_local_process_zero = self.is_local_process_zero()
  File "/usr/local/lib/python3.7/dist-packages/transformers/trainer.py", line 1784, in is_local_process_zero
    return self.args.local_process_index == 0
  File "/usr/local/lib/python3.7/dist-packages/transformers/file_utils.py", line 1605, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/transformers/training_args.py", line 864, in local_process_index
    return xm.get_ordinal(local=True)
TypeError: get_ordinal() got an unexpected keyword argument 'local'
```

## Expected behavior

The training should run without errors. I achieved this by simply replacing line 864 of /transformers/training_args.py:
```ruby
            return xm.get_ordinal(local=True)
```
with:
```ruby
            return xm.get_local_ordinal()
```
Following torch docs at:
            https://pytorch.org/xla/release/1.5/_modules/torch_xla/core/xla_model.html

If this is the correct syntax (and this behaviour is not due to something wrong in my environment), this easy fix should be enough. My model trained correctly.


Comments:
Thanks for the catch! Since you have the proper fix indeed, would like to make a PR with it?

Done, thanks!

Closed by #11922