Error: 'GPT2Model' object has no attribute '_step' when converting tf-based checkpoint into pytorch
Author: publicstaticvo
Pull request: no
Labels: name: wontfix

I'm trying to convert a tensorflow-based GPT-2 checkpoint into pytorch using `convert_gpt2_checkpoint_to_pytorch`, and get errors like:
```
INFO:transformers.modeling_gpt2:Converting TensorFlow checkpoint from /content/model.ckpt-220000
INFO:transformers.modeling_gpt2:Loading TF weight global_step with shape []
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/LayerNorm_embed_norm/beta with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/LayerNorm_embed_norm/beta/adafactor_v with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/LayerNorm_embed_norm/gamma with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/LayerNorm_embed_norm/gamma/adafactor_v with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/pos_embed with shape [1024, 1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/pos_embed/adafactor_vc with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/pos_embed/adafactor_vr with shape [1024]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/word_embed with shape [8021, 1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/word_embed/adafactor_vc with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/embeddings/word_embed/adafactor_vr with shape [8021]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer00/LayerNorm_mlp_ln0/beta with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer00/LayerNorm_mlp_ln0/beta/adafactor_v with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer00/LayerNorm_mlp_ln0/gamma with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer00/LayerNorm_mlp_ln0/gamma/adafactor_v with shape [1536]
...
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer47/value_layer/bias/adafactor_v with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer47/value_layer/kernel with shape [1536, 1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer47/value_layer/kernel/adafactor_vc with shape [1536]
INFO:transformers.modeling_gpt2:Loading TF weight newslm/layer47/value_layer/kernel/adafactor_vr with shape [1536]
---------------------------------------------------------------------------
ModuleAttributeError                      Traceback (most recent call last)
<ipython-input-38-45b704eacf86> in <module>()
      1 from transformers.convert_gpt2_original_tf_checkpoint_to_pytorch import convert_gpt2_checkpoint_to_pytorch
----> 2 convert_gpt2_checkpoint_to_pytorch('./model.ckpt-220000', '', 'pytorch')

2 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __getattr__(self, name)
    770                 return modules[name]
    771         raise ModuleAttributeError("'{}' object has no attribute '{}'".format(
--> 772             type(self).__name__, name))
    773 
    774     def __setattr__(self, name: str, value: Union[Tensor, 'Module']) -> None:

ModuleAttributeError: 'GPT2Model' object has no attribute '_step'
```
It seems that the program cannot convert `global_step` layer into pytorch. Is there any solution to this?


Comments:
same problem

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.