error with DataCollatorForLanguageModeling
Author: minmie
Pull request: no

### System Info

- `transformers` version: 4.36.2
- Platform: Linux-3.10.0-1160.99.1.el7.x86_64-x86_64-with-glibc2.17
- Python version: 3.10.13
- Huggingface_hub version: 0.20.2
- Safetensors version: 0.4.1
- Accelerate version: 0.25.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.1.0+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker and @younesbelkada

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I want to fine-tune gpt2 for text summarization, but an error occurs when I create the batch input with DataCollatorForLanguageModeling. My code is as follows:
```python
from transformers import AutoTokenizer, AutoModelForCausalLM, DataCollatorForTokenClassification, \
    DataCollatorWithPadding, DataCollatorForLanguageModeling, DataCollatorForSeq2Seq

model_path = "gpt2"

tokenizer = AutoTokenizer.from_pretrained(model_path)
# model = AutoModelForCausalLM.from_pretrained(model_path)
tokenizer.pad_token = tokenizer.eos_token
content = [
    "x x",
    "x x x"
]

summary = [
    "y y",
    "y y y"
]

batch = []

for c, s in zip(content, summary):
    sample_input_ids = tokenizer.encode('<content>' + c + '<summary>')
    label_input_ids = tokenizer.encode(s) + [tokenizer.eos_token_id]
    input_ids = sample_input_ids + label_input_ids
    labels = [-100] * len(sample_input_ids) + label_input_ids
    batch.append({"input_ids": input_ids, "labels": labels})

data_collator1 = DataCollatorForTokenClassification(tokenizer)
# data_collator2 = DataCollatorWithPadding(tokenizer)
data_collator3 = DataCollatorForLanguageModeling(tokenizer, mlm=False)
data_collator4 = DataCollatorForSeq2Seq(tokenizer)







# i used this collator to create batch input and an error occured.
print(data_collator3(batch))

"""
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (`labels` in this case) have excessive nesting (inputs type `list` where type `int` is expected).
"""

# I aslo try this two collator and there's an extra attention_mask returned，but gpt2 dose't need it.
# print(data_collator1(batch))
# print(data_collator4(batch))
"""
{'input_ids': tensor([[   27, 11299,    29,    87,  2124,    27, 49736,    29,    88,   331,
         50256, 50256, 50256],
        [   27, 11299,    29,    87,  2124,  2124,    27, 49736,    29,    88,
           331,   331, 50256]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0],
        [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]), 'labels': tensor([[ -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,    88,   331,
         50256,  -100,  -100],
        [ -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,    88,
           331,   331, 50256]])}
{'input_ids': tensor([[   27, 11299,    29,    87,  2124,    27, 49736,    29,    88,   331,
         50256, 50256, 50256],
        [   27, 11299,    29,    87,  2124,  2124,    27, 49736,    29,    88,
           331,   331, 50256]]), 'labels': tensor([[ -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,    88,   331,
         50256,  -100,  -100],
        [ -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,    88,
           331,   331, 50256]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0],
        [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]])}
"""


print(1)
```


### Expected behavior

Considering I am fine-tuning gpt2 for text summarization, I expect to get a batch input like this:

```
{
'input_ids': tensor([[   27, 11299,    29,    87,  2124,    27, 49736,    29,    88,   331, 50256, 50256, 50256],
        [   27, 11299,    29,    87,  2124,  2124,    27, 49736,    29,    88, 331,   331, 50256]]), 
'labels': tensor([[ -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,    88,   331,50256,  -100,  -100],
        [ -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,  -100,    88,331,   331, 50256]])
}



```

Comments:
Hey 🤗  thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co/) instead? I'm sure the community will be of help!

Thanks!