block sparse bert
Author: pingpiang2019
Pull request: no
Labels: name: wontfix

I got following error while running example usage from https://huggingface.co/madlag/bert-base-uncased-squad1.1-block-sparse-0.09-ampere-v1

Do I need specific torch or transformers setup? Thanks in advance!


Some weights of BertModel were not initialized from the model checkpoint at madlag/bert-base-uncased-squad1.1-block-sparse-0.09-ampere-v1 and are newly initialized: ['bert.pooler.dense.weight', 'bert.pooler.dense.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Traceback (most recent call last):
  File "test.py", line 6, in <module>
    tokenizer="madlag/bert-base-uncased-squad1.1-block-sparse-0.09-ampere-v1"
  File "/workspace/transformers/src/transformers/pipelines.py", line 3231, in pipeline
    framework = framework or get_framework(model)
  File "/workspace/transformers/src/transformers/pipelines.py", line 107, in get_framework
    model = AutoModel.from_pretrained(model, revision=revision)
  File "/workspace/transformers/src/transformers/models/auto/modeling_auto.py", line 698, in from_pretrained
    pretrained_model_name_or_path, *model_args, config=config, **kwargs
  File "/workspace/transformers/src/transformers/modeling_utils.py", line 1156, in from_pretrained
    model.__class__.__name__, "\n\t".join(error_msgs)
RuntimeError: Error(s) in loading state_dict for BertModel:
	size mismatch for bert.encoder.layer.0.attention.self.query.weight: copying a param with shape torch.Size([768, 768]) from checkpoint, the shape in current model is torch.Size([256, 768]).
	size mismatch for bert.encoder.layer.0.attention.self.query.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([256]).
	size mismatch for bert.encoder.layer.0.attention.self.key.weight: copying a param with shape torch.Size([768, 768]) from checkpoint, the shape in current model is torch.Size([256, 768]).
	size mismatch for bert.encoder.layer.0.attention.self.key.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([256]).
	size mismatch for bert.encoder.layer.0.attention.self.value.weight: copying a param with shape torch.Size([768, 768]) from checkpoint, the shape in current model is torch.Size([256, 768]).

Comments:
I can produce your error. @madlag running the following code:

```python
from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="madlag/bert-base-uncased-squad1.1-block-sparse-0.09-ampere-v1",
    tokenizer="madlag/bert-base-uncased-squad1.1-block-sparse-0.09-ampere-v1"
)

predictions = qa_pipeline({
    'context': "Frédéric François Chopin, born Fryderyk Franciszek Chopin (1 March 1810 – 17 October 1849), was a Polish composer and virtuoso pianist of the Romantic era who wrote primarily for solo piano.",
    'question': "Who is Frederic Chopin?",
})

print(predictions)
```

results in the above error. Any ideas on how to fix it?

It looks like there is a bug with the "ampere optimized"  models I uploaded, thank you for your feedback, I will check what is happening.
Right now I would advise you to use the non ampere ones (like madlag/bert-base-uncased-squad1.1-block-sparse-0.13-v1 ), the ampere version is not really good at this time.
I am working on this, there should be some new, better and faster models soon, non-ampere optimized ones, then ampere optimized a bit latter.


This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.