Running GLUE example failed since Apr 17
Author: jingyanwangms
Pull request: no

### System Info

We have continuous monitoring that runs latest huggingface models to benchmark performance, and below script failed since Apr 17
python -m torch.distributed.launch --nproc_per_node=8   /workspace/transformers/examples/pytorch/text-classification/run_glue.py --model_name_or_path microsoft/deberta-large  --task_name MRPC --max_seq_length 128 --learning_rate 3e-6 --do_train --output_dir /dev/shm --overwrite_output_dir --max_steps 200 --logging_steps 20 --per_device_train_batch_size 32 --fp16  

It should be a check in between Apr 16 9:33 PM and Apr 17 4:01 PM PST.

torch                        1.14.0.dev20221213+cu116
hugginface install from source at whatever timestamp

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Will add conda environment info later
python -m torch.distributed.launch --nproc_per_node=8   /workspace/transformers/examples/pytorch/text-classification/run_glue.py --model_name_or_path microsoft/deberta-large  --task_name MRPC --max_seq_length 128 --learning_rate 3e-6 --do_train --output_dir /dev/shm --overwrite_output_dir --max_steps 200 --logging_steps 20 --per_device_train_batch_size 32 --fp16  

### Expected behavior

above program succeed, instead here's the error
Traceback (most recent call last):
  File "/workspace/transformers/examples/pytorch/text-classification/run_glue.py", line 626, in <module>
    main()
  File "/workspace/transformers/examples/pytorch/text-classification/run_glue.py", line 217, in main
    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/transformers-4.29.0.dev0-py3.8.egg/transformers/hf_argparser.py", line 332, in parse_args_into_dataclasses
    obj = dtype(**inputs)
  File "<string>", line 110, in __init__
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/transformers-4.29.0.dev0-py3.8.egg/transformers/training_args.py", line 1255, in __post_init__
    and (self.device.type != "cuda")
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/transformers-4.29.0.dev0-py3.8.egg/transformers/training_args.py", line 1615, in device
    return self._setup_devices
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/transformers-4.29.0.dev0-py3.8.egg/transformers/utils/generic.py", line 54, in __get__
    cached = self.fget(obj)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/transformers-4.29.0.dev0-py3.8.egg/transformers/training_args.py", line 1549, in _setup_devices
    self.distributed_state = PartialState(backend=self.xpu_backend)
  File "/opt/conda/envs/ptca/lib/python3.8/site-packages/accelerate/state.py", line 129, in __init__
    torch.distributed.init_process_group(backend="nccl", **kwargs)
TypeError: init_process_group() got multiple values for keyword argument 'backend'

Comments:
This is the commit that seems to cause the issue: https://github.com/huggingface/transformers/commit/03462875cc2d6506eb66a74de7d19b93ce968596

@jingyanwangms Thanks for raising this issue! 

There was an issue that occurred on the development branch with the introduction of PartialState from accelerate and was reported here: #22816, which is likely related. Could you share more information about the running environment, specifically sharing the output of running `transformers-cli env`?

For me, updating `accelerate` via `pip install git+https://github.com/huggingface/accelerate ` solved it.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.