tensor size mismatch in NER.py
Author: TahaAslani
Pull request: no

Hi,

I am trying to classify tokens for a task similar to NER. I am using the following code:

transformers/examples/pytorch/token-classification/run_ner.py
 
My input data is in JSON format for both train and test. When I perform classification using BioBERT, it runs without any issue. When I use BERT base (cased or uncased), I get the following error in the middle of the training:

Traceback (most recent call last):
  File "/home/tv349/PharmaBERT/transformers/examples/pytorch/token-classification/run_ner.py", line 530, in <module>
    main()
  File "/home/tv349/PharmaBERT/transformers/examples/pytorch/token-classification/run_ner.py", line 463, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/home/tv349/.conda/envs/HF/lib/python3.8/site-packages/transformers/trainer.py", line 1269, in train
    tr_loss += self.training_step(model, inputs)
  File "/home/tv349/.conda/envs/HF/lib/python3.8/site-packages/transformers/trainer.py", line 1754, in training_step
    loss = self.compute_loss(model, inputs)
  File "/home/tv349/.conda/envs/HF/lib/python3.8/site-packages/transformers/trainer.py", line 1786, in compute_loss
    outputs = model(**inputs)
  File "/home/tv349/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/tv349/.conda/envs/HF/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py", line 1712, in forward
    outputs = self.bert(
  File "/home/tv349/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/tv349/.conda/envs/HF/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py", line 984, in forward
    embedding_output = self.embeddings(
  File "/home/tv349/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/tv349/.conda/envs/HF/lib/python3.8/site-packages/transformers/models/bert/modeling_bert.py", line 221, in forward
    embeddings += position_embeddings
RuntimeError: The size of tensor a (607) must match the size of tensor b (512) at non-singleton dimension 1

Also, this is how I call the function:

python transformers/examples/pytorch/token-classification/run_ner.py \
  --model_name_or_path $model_path \
  --train_file $train_file \
  --validation_file $validation_file \
  --output_dir $root_output_dir \
  --overwrite_output_dir \
  --do_train \
  --do_eval

I don't think the problem is in my input data or setting because everything works for BioBERT, but failed for BERT base.

Could you please let me know what can I do to avoid this issue?

Thanks!

Comments:
cc @sgugger 

Mmm the problem looks like it's coming from the position embedding with an input of size 607, which is larger than 512 (the maximum handled apparently).

Thanks for the response. But why do you think it works for BioBert and not BERT base?

Will it help if I provide my training data?

Anything that could helps us reproduce the issue would help, yes.

> But why do you think it works for BioBert and not BERT base?

It seems this is happening due to the [`model_max_length`] property in `tokenizer_config.json`. So far I could understand from the [docs], setting this property to a value will trim the tokens while tokenization (please, correct me if I am wrong).

[`tokenizer_config.json` of biobert] sets `model_max_length=512`. But in case of bert-base [cased] or [uncased], `model_max_length` is not specified.

[`model_max_length`]: https://huggingface.co/transformers/internal/tokenization_utils.html?highlight=model_max_length#transformers.tokenization_utils_base.PreTrainedTokenizerBase
[docs]: https://huggingface.co/transformers/internal/tokenization_utils.html?highlight=model_max_length#transformers.tokenization_utils_base.PreTrainedTokenizerBase
[`tokenizer_config.json` of biobert]: https://huggingface.co/dmis-lab/biobert-v1.1/blob/main/tokenizer_config.json
[cased]: https://huggingface.co/bert-base-cased/blob/main/tokenizer_config.json
[uncased]: https://huggingface.co/bert-base-uncased/blob/main/tokenizer_config.json

Yes, that is very likely the case. I think the script is simply missing a `max_seq_length` parameter (like there is for text classification for instance) that the user should set when the `model_max_length` is either too large or simply missing.

I will add that tomorrow.

Adding model_max_length=512 to the tokenizer_config.json solved the issue. Thank you very much!

@sgugger that would be a great idea. I was looking for that option but couldn't find it. Thank you very much for your consideration.