albertModel object has no attribute bias
Author: Pydataman
Pull request: no

transformers version:2.9.0

model = AlbertModel.from_pretrained("xxx", from_tf=True)

Initialize PyTorch weight ['albert', 'encoder', 'albert_layer_groups', '0', 'albert_layers', '0', 'attention', 'key', 'kernel'] from bert/encoder/transformer/group_0/inner_group_0/attention_1/self/key/kernel
Initialize PyTorch weight ['albert', 'encoder', 'albert_layer_groups', '0', 'albert_layers', '0', 'attention', 'query', 'bias'] from bert/encoder/transformer/group_0/inner_group_0/attention_1/self/query/bias
Initialize PyTorch weight ['albert', 'encoder', 'albert_layer_groups', '0', 'albert_layers', '0', 'attention', 'query', 'kernel'] from bert/encoder/transformer/group_0/inner_group_0/attention_1/self/query/kernel
Initialize PyTorch weight ['albert', 'encoder', 'albert_layer_groups', '0', 'albert_layers', '0', 'attention', 'value', 'bias'] from bert/encoder/transformer/group_0/inner_group_0/attention_1/self/value/bias
Initialize PyTorch weight ['albert', 'encoder', 'albert_layer_groups', '0', 'albert_layers', '0', 'attention', 'value', 'kernel'] from bert/encoder/transformer/group_0/inner_group_0/attention_1/self/value/kernel
Initialize PyTorch weight ['albert', 'encoder', 'albert_layer_groups', '0', 'albert_layers', '0', 'ffn', 'bias'] from bert/encoder/transformer/group_0/inner_group_0/ffn_1/intermediate/dense/bias
Initialize PyTorch weight ['albert', 'encoder', 'albert_layer_groups', '0', 'albert_layers', '0', 'ffn', 'kernel'] from bert/encoder/transformer/group_0/inner_group_0/ffn_1/intermediate/dense/kernel
Initialize PyTorch weight ['albert', 'encoder', 'albert_layer_groups', '0', 'albert_layers', '0', 'ffn_output', 'bias'] from bert/encoder/transformer/group_0/inner_group_0/ffn_1/intermediate/output/dense/bias
Initialize PyTorch weight ['albert', 'encoder', 'albert_layer_groups', '0', 'albert_layers', '0', 'ffn_output', 'kernel'] from bert/encoder/transformer/group_0/inner_group_0/ffn_1/intermediate/output/dense/kernel
Initialize PyTorch weight ['albert', 'pooler', 'bias'] from bert/pooler/dense/bias
Initialize PyTorch weight ['albert', 'pooler', 'kernel'] from bert/pooler/dense/kernel
Traceback (most recent call last):
  File "d:\python_workbase\project\transformers_test\test.py", line 7, in <module>
    model = AlbertModel.from_pretrained("D:\work\model\\albert_tiny_zh_google", from_tf=True)
  File "D:\Programs\Python\Python37\lib\site-packages\transformers\modeling_utils.py", line 640, in from_pretrained
    model = cls.load_tf_weights(model, config, resolved_archive_file[:-6])  # Remove the '.index'
  File "D:\Programs\Python\Python37\lib\site-packages\transformers\modeling_albert.py", line 139, in load_tf_weights_in_albert
    pointer = getattr(pointer, "bias")
  File "D:\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py", line 591, in __getattr__
    type(self).__name__, name))
AttributeError: 'AlbertModel' object has no attribute 'bias'

Comments:
Hi! What is the `xxx` model? Is it one of our pre-trained checkpoints, is it an original TF checkpoint? Did this error happen in previous versions? Would you mind giving a bit of context?

Hi, I tried this model https://storage.googleapis.com/albert_models/albert_base_zh.tar.gz , which is new model release by Google on 2019 Dec. 30 on Albert's official [github page](https://github.com/google-research/albert). And I encountered the same error. My code is:
```Python
from transformers import AlbertModel, AlbertConfig
config = json.load(open('albert_base/albert_config.json'))
config = AlbertConfig(**config)
model = AlbertModel.from_pretrained('albert_base/model.ckpt-best', config=config, from_tf=True)
```


Thank you!

Thanks, I'll take a look.

That's because you're trying to load a checkpoint without first converting it. You should run the conversion script under `src/transformers/convert_albert_original_tf_checkpoint_to_pytorch.py`:

```
python src/transformers/convert_albert_original_tf_checkpoint_to_pytorch.py \
    --tf_checkpoint_path=$PATH_TO_ALBERT/albert_base_chinese/model.ckpt-best \
    --albert_config_file=$PATH_TO_ALBERT//albert_base_chinese/albert_config.json \
    --pytorch_dump_path=$PATH_TO_ALBERT/albert_chinese.pt 
```

It works, thank you!