Issue 1:
Environment info

transformers version: 4.0.0 --rc1

Platform: Colab

Python version: 3.6.9

PyTorch version (GPU?): TESLA V4

Tensorflow version (GPU?): 2.3.0

Using GPU in script?: YES

Using distributed or parallel set-up in script?: NO

Who can help

Information

Model I am using (Bert, XLNet ...): T5-1.1

The problem arises when using:

the official example scripts: (give details below)

[X ] my own modified scripts: (give details below)

from transformers import T5TokenizerFast, T5ForConditionalGeneration
TOKENIZER_NAME = 't5-base'
MODEL_NAME = 'google/t5-v1_1-base'
tokenizer = T5TokenizerFast.from_pretrained(MODEL_NAME)
special_tokens_dict = {'additional_special_tokens': ['<ORG>','<PERSON>']}
num_added_toks = tokenizer.add_special_tokens(special_tokens_dict)
model = T5ForConditionalGeneration.from_pretrained('google/t5-v1_1-base', return_dict=True)
model.resize_token_embeddings(len(tokenizer))
model.to("cuda")

The tasks I am working on is:

an official GLUE/SQUaD task: (give the name)

my own task or dataset: (give details below)

Custom dataset

To reproduce

Steps to reproduce the behavior:

Attempting to add Entity tokens to T5 1.1, upon loading from pretrained the following error occurs:

size mismatch for lm_head.weight: copying a param with shape torch.Size([32128, 768]) from checkpoint, the shape in current model is torch.Size([32102, 768]).

I am assuming the addition of the special tokens did not get propagated to the lm head size.

I would expect the LM Head to be resized in addition to the standard layers.

Many Thanks,

Chris

Issue 2:
Environment info

Google colab

Who can help

T5: @patrickvonplaten

Information

I'm noticing something strange with T5. The model embedding size and the tokenizer size does not match.

When I try to resize the model to have a smaller embedding this crashes CUDA. This is probably two bugs - one for the size mismatch, and one for shortening the embedding causing a crash.

from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("t5-base")
print (len(tokenizer))
model = AutoModel.from_pretrained("t5-base")
print (model.shared)
model.resize_token_embeddings(len(tokenizer))
model.to('cuda')

Expected behavior

Expected behaviour is regular loading of the model into cuda.

What I got instead was:

32100

Some weights of T5Model were not initialized from the model checkpoint at t5-base and are newly initialized: ['encoder.embed_tokens.weight', 'decoder.embed_tokens.weight']

You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

Embedding(32128, 768)

RuntimeError Traceback (most recent call last)

in ()

5 print (model.shared)

6 model.resize_token_embeddings(len(tokenizer))

----> 7 model.to('cuda')

3 frames

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in convert(t)

608 if convert_to_format is not None and t.dim() == 4:

609 return t.to(device, dtype if t.is_floating_point() else None, non_blocking, memory_format=convert_to_format)

--> 610 return t.to(device, dtype if t.is_floating_point() else None, non_blocking)

611

612 return self._apply(convert)

RuntimeError: CUDA error: device-side assert triggered