Issue 1:
🐛 Bug

Information

Model I am using (Bert, XLNet ...): BART

Language I am using the model on (English, Chinese ...): English

The problem arises when using:

the official example scripts: (give details below)

my own modified scripts: (give details below)

The tasks I am working on is:

an official GLUE/SQUaD task: (give the name): CNN/DM

my own task or dataset: (give details below)

To reproduce

Hi @sshleifer,

Thanks for the amazing model!

I found a bug when alternatively trying to use forward to train the BartForConditionalGeneration and use generate to inference and evaluate the trained model. As shown in the code below, if I first use generate function then call forward function, the generation_mode attribute of decoder is set to True, and the shape of the decoder_output seems incorrect.

model = BartForConditionalGeneration.from_pretrained('bart-large-cnn')
tokenizer = BartTokenizer.from_pretrained('bart-large-cnn')

# input sequence
input_seq = "Bart is a deep-learning pretrained model implemented in pytorch. It can smoothly handle summarization. It is a big model pretrained for generation tasks, especially summarizaition."
input_ids = torch.LongTensor([tokenizer.encode(input_seq)])

# expected output sequence
decoder_input_seq = "Bart is a big pretrained deep model in pytorch for summarization."
decoder_input_ids = torch.LongTensor([tokenizer.encode(decoder_input_seq)])

# using generate method to inference
with torch.no_grad():
result = model.generate(input_ids=input_ids, eos_token_ids=tokenizer.eos_token_id, num_beams=4, max_length=20)
print(tokenizer.decode(result[0]))
# 'B. It is a big model pretrained for generation tasks, especially summarizaition. It'

# NOW use forward to train
result = model(input_ids, decoder_input_ids=decoder_input_ids)
# the shape of decoder_output and encoder_output
# what expected is: <1, 18, 50264> and <40, 1, 1024>
# but actual output is: torch.Size([1, 1, 50264]) torch.Size([40, 1, 1024])
print(result[0].shape, result[2].shape)

Such issue can seemingly be addressed by mannually setting the generation_mode.

# mannually set the generation mode to False **seemingly** fix the issue
model.model.decoder.generation_mode=False
result = model(input_ids, decoder_input_ids=decoder_input_ids)
print(result[0].shape, result[2].shape)
# output is: torch.Size([1, 18, 50264]) torch.Size([40, 1, 1024]) and make sense

However, now the output of the forward function doesn't make sense, as shown below.

with torch.no_grad():
result = model.generate(input_ids=input_ids, eos_token_ids=tokenizer.eos_token_id, num_beams=4, max_length=20)

model.model.decoder.generation_mode=False
new_result = model(input_ids, decoder_input_ids=result)

print(tokenizer.decode(torch.argmax(new_result[0][0], dim=1)))
# ' and and and and and and and and and and and and and and and and and and and'

In expectation, when we feed the generated decoder sequence to the decoder and remain the input of the encoder unchanged, the output of the decoder would at least resemble the decoder_input. However, the actual output of the model is ' and and and and and and and and and and and and and and and and and and and', which is definately not a reasonable output of a trained decoder.

To go deeper, I tested different samples, all feeding the generated output of a encoder_input to the decoder and inspect the decoder_output (which "should" resemble decoder_input, i.e. generated sequence), the same happens everytime for me (most output is the duplication of a single token for many times, like "the the the ...", "and and and...", ". . . . ").

Given all these results, I think there is a bug in either the implementation of generate or that of forward, the revert of generation_mode is simple, but I don't think the output of forward is a reasonable result currently. Could you please look into the issue?

@sshleifer If I'm not using those functions in a correct manner, any advise or instruction is welcomed! Many thanks for the help!

Environment info

transformers version: master-branch

Platform: windows 10

Python version: 3.7.0

PyTorch version (GPU?): 1.4.0

Tensorflow version (GPU?): /

Using GPU in script?: Yes

Using distributed or parallel set-up in script?: No

Issue 2:
❓ Questions & Help

Hi there,

when I load the pretrained Camenbert model and tokenizer via

model = CamembertForMaskedLM.from_pretrained('camembert-base') tokenizer = CamembertTokenizer.from_pretrained('camembert-base')

the length of the tokenizer is 32004 but the vocab_size of the model is 32005.

print(len(tokenizer))

'print(model.config.vocab_size'

This throws me an error

Index out of range

when I try to adapt the lm_finetuning example because of

model.resize_token_embeddings(len(tokenizer))

It runs when I comment out this line. So my question is, is this the intended behaviour resp. what's the reason for the unevenness between the tokenizer and the model vocab_size?