BERT's Embedding/Vocab Size in Code is Different from Provided Pretrained Config
Author: LFhase
Pull request: no

## 🐛 A Subtle Bug

Hi, I really appreciate your work but I found a subtle problem here. Could you take a look of it?

- The model I am using is **BERT**.

- The language I am using the model on is **English**.

- The problem arises when using:

- The task I am working on is to simply initialize a BERT object with my own modifications to config, i.e., `BertConfig` class.

## To Reproduce

Steps to reproduce the behavior:

Simplely run this line:
```
BertModel.from_pretrained("bert-base-cased",config=BertConfig(output_hidden_states=True))
```
Then we have following error message:
`
File "D:\Anaconda3\envs\gnner\lib\site-packages\transformers\modeling_utils.py", line 486, in from_pretrained
    model.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for BertModel:
	size mismatch for bert.embeddings.word_embeddings.weight: copying a param with shape torch.Size([28996, 768]) from checkpoint, the shape in current model is torch.Size([30522, 768]).
`

## Expected behavior

It should be good to run instead of reporting issues like that.

## Possible reason
The issue is because of `line 86` in `configuration_bert.py`, where the vocabulary size is **`30522`**. The default vocabulary size I believe should be consistent with that in the config file, i.e., `https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-config.json`, where it's **`28996`**.

## Environment

* OS: Windows 10
* Python version: 3.7.3
* PyTorch version: 1.3.0
* PyTorch Transformers version (or branch): latest pip package
* Using GPU ? It's independent of environments like this I believe.


<!-- Add any other context about the problem here. -->

Comments:
The default configuration in `configuration_bert.py` is for `bert-base-uncased` model. I am not sure what you are trying to do here will work or not but here is what I would suggest try doing:

First Load configuration manually from `bert-base-case` json. Then change the parameters you want to change and then pass it to `from_pretrained` function

> The default configuration in `configuration_bert.py` is for `bert-base-uncased` model. I am not sure what you are trying to do here will work or not but here is what I would suggest try doing:
> 
> First Load configuration manually from `bert-base-case` json. Then change the parameters you want to change and then pass it to `from_pretrained` function

Thank you NaxAlpha for your immediate reply!

My intention is just simply to get outputs of all hidden layers from a pre-trained BERT and found this 'issue'. Your solution sounds good! 

In the future, might it be better to load the corresponding config according to the input parameter, i.e., the string like `bert-base-uncased` or `bert-base-cased`, since the weights are also loaded according to this string?

Great. I have verified that it is working:

https://colab.research.google.com/drive/1IPgcACm38dIUaj9RqTWw9xbwwywIOpXf

As @NaxAlpha says, the default parameters are that of the `bert-base-uncased` model. If you wish to instantiate a `BertConfig` from the `bert-base-cased` model  with the `output_hidden_states` flag set to `True`, you would do it as follows:

```py
config = BertConfig.from_pretrained("bert-base-cased", output_hidden_states=True)
model = BertModel.from_pretrained("bert-base-cased", config=config)
```

Thanks, guys.
Your replies solve my question well. 

I am on Ubuntu  where also reports 
```
Error(s) in loading state_dict for BertForSequenceClassification:
	size mismatch for bert.embeddings.word_embeddings.weight: copying a param with shape torch.Size([30522, 768]) from checkpoint, the shape in current model is torch.Size([28996, 768]).
```
I am currently checking solutions above.

> I am on Ubuntu where also reports
> 
> ```
> Error(s) in loading state_dict for BertForSequenceClassification:
> 	size mismatch for bert.embeddings.word_embeddings.weight: copying a param with shape torch.Size([30522, 768]) from checkpoint, the shape in current model is torch.Size([28996, 768]).
> ```
> 
> I am currently checking solutions above.

Hi, I think it may be better to also post the minimal code to reproduce the issue here. 

Was facing a similar issue previously as I tried to adapt allenai/scibert_scivocab_cased model. I was previously still using the bert config.json. By making sure the config.json matches the model I am using (in my case was the scibert config), was able to bypass this issue. 

tokenizer = BertTokenizer.from_pretrained("../models/bert_large_cased_tokenizer/vocab.json")
model = BertModel.from_pretrained("../models/bert_large_cased_model/pytorch_model.bin",config="../models/bert_large_cased_model/config.json")