DataCollatorForTokenClassification pads labels incorrectly for LukeModel
Author: SuijkerbuijkP
Pull request: no

### System Info

- `transformers` version: 4.26.0
- Platform: Linux-4.18.0-348.12.2.el8_5.x86_64-x86_64-with-glibc2.28
- Python version: 3.9.16
- Huggingface_hub version: 0.12.0
- PyTorch version (GPU?): 1.13.1+cu117 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I am using a LUKE based workflow with my own text dataset. This works perfectly fine using a standard workflow (e.g. the pretrained luke-large-finetuned-conll-2003 LukeTokenizer with padding and truncation in combination with a standard Trainer instance), but when trying to implement dynamic padding strange behavior was observed. Still using the same tokenizer, but now only with truncation, and then using the DataCollatorForTokenClassification to pad batches during Trainer.train(), the batch size was reportedly wrong (ValueError: Expected input batch_size (30) to match target batch_size (46)). 

When comparing the output of the working workflow with padding and truncation in the Tokenizer, it is observed that the labels, entity_ids, entity_position_ids, entity_start_positions, entity_end_positions and entity_attention_mask are of the same size, and the input_ids and attention_mask are also of the same (but possibly different) size. As mentioned, this works.

When checking the output after the DataCollatorForTokenClassification, the labels are the same size as the input_ids and attention_mask. This is incorrect for the selected tokenizer, and makes it such that the mentioned error is given. 

### Expected behavior

labels are padded according to the entity_ids, not according to the input_ids. 

Comments:
Did you tried the `DataCollatorForLukeTokenClassification` :thinking: 

For LUKE and NER there's this demo project available:

https://github.com/huggingface/transformers/tree/main/examples/research_projects/luke

Thanks, I did not know that exists, as it is not part of the normal datacollators. Good chance that that works.

Any chance this will become part of the transformers library? Instead of just importing it we have to copy over the utils file. 

No this is too specific to be in the library proper.

The data collator in the script pointed out by @stefan-it works, when removing the original_entity_spans part of that collator that is not used or outputted by the LukeTokenizer. So thanks for that! 

Is it too specific? All the other luke parts are available as imports, having this one thing only available as a custom code import feels as a lost opportunity. 

Anyways, thanks for the quick help!

Keep in mind that the Transformers library is primarily a library of models, not data collators ;-)

Good point, you tend to forget that when the whole solution works so seamlessly :) I guess I was spoiled ;)