Problem with c10d pickle in distributed run
Author: jmzeng
Pull request: no

### System Info

- `transformers` version: 4.32.1
- Platform: Linux-5.15.0-83-generic-x86_64-with-glibc2.31
- Python version: 3.10.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: 0.22.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.0.1+cu118 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help?

@pacman100 @muellerzr I'm experiencing an issue where there is a pickle error with multi-node distributed training. The error would happen about 190 steps into training and I've previously trained the same data for over 300 steps on a single node. I actually also hit this bug when using the latest transformers version from the main branch. 

See error below:
```python
54 Traceback (most recent call last):
[...]
train_llm.py", line 213, in pretrain
60     trainer.train()
61   File "/usr/lib/python3/dist-packages/transformers/trainer.py", line 1555, in train
62     return inner_training_loop(
63   File "/usr/lib/python3/dist-packages/transformers/trainer.py", line 1815, in _inner_training_loop
64     for step, inputs in enumerate(epoch_iterator):
65   File "/usr/lib/python3/dist-packages/accelerate/data_loader.py", line 593, in __iter__
66     next_batch, next_batch_info = self._fetch_batches(main_iterator)
67   File "/usr/lib/python3/dist-packages/accelerate/data_loader.py", line 534, in _fetch_batches
68     broadcast_object_list(batch_info)
69   File "/usr/lib/python3/dist-packages/accelerate/utils/operations.py", line 456, in broadcast_object_list
70     torch.distributed.broadcast_object_list(object_list, src=from_process)
71   File "/usr/lib/python3/dist-packages/torch/distributed/distributed_c10d.py", line 1451, in wrapper
72     return func(*args, **kwargs)
73   File "/usr/lib/python3/dist-packages/torch/distributed/distributed_c10d.py", line 2277, in broadcast_object_list
74     object_list[i] = _tensor_to_object(obj_view, obj_size)
75   File "/usr/lib/python3/dist-packages/torch/distributed/distributed_c10d.py", line 1970, in _tensor_to_object
76     return _unpickler(io.BytesIO(buf)).load()
77 _pickle.UnpicklingError: invalid load key, '\x01'.
```

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

The run is launched using 
`torchrun --nnodes=2 --node_rank=1 --nproc-per-node=8 --rdzv-id=428 --rdzv-backend=c10d --rdzv-endpoint=IP1:1234 train_llm.py`

I am using the SFTTrainer with the ConstantLengthDataset and `packing=True`. 

### Expected behavior

I expect this should continue to train. 

Comments:
cc @muellerzr  as it is related to dataloaders.

Same comment as here: https://github.com/huggingface/trl/issues/796#issuecomment-1735049375

Can you give us some details about your dataset? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.