AttributeError: 'GPT2LMHeadModel' object has no attribute 'get_encoder'
Author: goji-patai
Pull request: no

Hi,
    I am trying to generate text from a  GPT2 model I have trained from scratch using custom english language data.
	OS: Windows 10
	transformers 3.5.0
	Pytorch 1.4.0 (upgrading torch did not help)
	Tensorflow 2.2.0

	GPT2LMHeadModel was trained using run_language_modeling.py with the following config
		{
  "_num_labels": 2,
  "activation_function": "gelu_new",
  "architectures": [
    "GPT2LMHeadModel"
  ],
  "attn_pdrop": 0.1,
  "bos_token_id": 50256,
  "do_sample": "false",
  "early_stopping": "false",
  "embd_pdrop": 0.1,
  "eos_token_id": 50256,
  "gradient_checkpointing": false,
  "initializer_range": 0.02,
  "is_decoder": "false",
  "is_encoder_decoder": "false",
  "layer_norm_epsilon": 1e-05,
  "model_type": "gpt2",
  "n_ctx": 1024,
  "n_embd": 768,
  "n_head": 12,
  "n_inner": null,
  "n_layer": 12,
  "n_positions": 1024,
  "output_attentions": "false",
  "output_hidden_states": "false",
  "output_past": "true",
  "resid_pdrop": 0.1,
  "summary_activation": null,
  "summary_first_dropout": 0.1,
  "summary_proj_to_labels": "true",
  "summary_type": "cls_index",
  "summary_use_proj": "true",
  "torchscript": "false",
  "use_bfloat16": "false",
  "vocab_size": 49051
}

  Training ended without errors.  I tried to generate text from the model using the following script(https://dejanbatanjac.github.io/gpt2-example/):
  
import random
import numpy as np
import torch
from transformers import GPT2LMHeadModel, GPT2Tokenizer

gpt2_model = GPT2LMHeadModel.from_pretrained("LOCAL PATH TO MODEL")
gpt2_tokenizer = GPT2Tokenizer.from_pretrained("LOCAL PATH TO MODEL TOKENIZER")
seed = random.randint(0, 13)
np.random.seed(seed)
torch.random.manual_seed(seed)
torch.cuda.manual_seed(seed)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

text = """Sample prompt text """

input_ids = torch.tensor(gpt2_tokenizer.encode(text, add_special_tokens=True)).unsqueeze(0) # bs=1

gpt2_model.to(device)
gpt2_model.eval()

outputs = gpt2_model.generate(
    input_ids.to(device), 
    max_length=500,
    do_sample=True,
    top_k=15,
    temperature=0.65
    )
print(gpt2_tokenizer.decode(outputs[0], skip_special_tokens=True))
outputs.shape,outputs[0].shape # (torch.Size([1, 500]), torch.Size([500]))

	Running the above generation script, I get the following error trace
2021-06-01 22:38:44.602457: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
Traceback (most recent call last):
  File "GPT2_Text_gen_4.py", line 26, in <module>
    temperature=0.65
  File "C:\Users\gojeb\anaconda3\envs\TF_Pytorch_Transformers\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\gojeb\anaconda3\envs\TF_Pytorch_Transformers\lib\site-packages\transformers\generation_utils.py", line 462, in generate
    model_kwargs = self._prepare_encoder_decoder_kwargs_for_generation(input_ids, model_kwargs)
  File "C:\Users\gojeb\anaconda3\envs\TF_Pytorch_Transformers\lib\site-packages\transformers\generation_utils.py", line 80, in _prepare_encoder_decoder_kwargs_for_generation
    encoder = self.get_encoder()
  File "C:\Users\gojeb\anaconda3\envs\TF_Pytorch_Transformers\lib\site-packages\torch\nn\modules\module.py", line 948, in __getattr__
    type(self).__name__, name))
AttributeError: 'GPT2LMHeadModel' object has no attribute 'get_encoder'

Please point me in the correct direction to solve this problem.

Thank you

Comments:
Hi there,
this is because in the config the value `is_encoder_decoder` is actually a string `"False"` which evaluates to `True` in python, and hence `generate` treats this model as an encoder-decoder model. `is_encoder_decoder` should be set to boolean false.

Ahh! Thank you.