scibert-nli out of dace
Author: dbsousa01
Pull request: no

## Environment info
     
- `transformers` version: 3.1.0
- Platform: macOS-10.15.5-x86_64-i386-64bit
- Python version: 3.8.5
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
Owner: @gsarti
Bert owner: @LysandreJik

## Information

By importing the model [here](https://huggingface.co/gsarti/scibert-nli?), two warnings prompt up

## To reproduce

Steps to reproduce the behavior:

1. Just import the model from the pretrained like in the example link

`FutureWarning: The class AutoModelWithLMHead is deprecated and will be removed in a future version. Please use AutoModelForCausalLM for causal language models, AutoModelForMaskedLM for masked language models and AutoModelForSeq2SeqLM for encoder-decoder models.`

`Some weights of BertForMaskedLM were not initialized from the model checkpoint at gsarti/scibert-nli and are newly initialized: ['cls.predictions.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.predictions.decoder.bias', 'cls.predictions.transform.dense.weight']`

## Expected behavior

Import without any kind of errors/warnings. I suppose the first warning is due to deprecation and should be solved by importing `AutoModelForMaskedLM` instead (just looking for confirmation and giving the heads-up). The second, it seems that some layers are out of date and not trained, it would be good if the owner could update the model if possible.


Comments:
Hi @dbsousa01 ,

Thanks for the information. The models I published on the HuggingFace model repository were produced using transformers 2.7.0 and tested up to version 2.11.0. Can you try using transformers 2.11.0 for your specific use case, to see if the same warnings are still present? 

Sadly, currently I am not in a situation that allows me to work on an update for the model, but using 2.11.0 may be enough for you! Otherwise you can try to replicate the model yourself by following the procedure described [here](https://github.com/gsarti/covid-papers-browser), using parameters specified in the model card [here](https://huggingface.co/gsarti/scibert-nli).

Hope this helps,

Gabriele

Hey @gsarti,

Thanks for the help. If I go back to `version 2.11.0` some other dependencies can break the model itself since some functions are already deprecated. 

Nevertheless, and I don't know if this makes sense, I downloaded your model from the older version (`2.11.0`) and imported as a pre-trained local model in the newer version (`3.1.0`). The warning no longer appears and the model's output seems to be stable, always giving the same results, which didn't happen when downloading the model with the most recent transformers version (`3.1.0`).

That's an interesting behavior though, I wouldn't expect a downloaded model to behave differently from a hosted one during the `from_pretrained` call. @LysandreJik @julien-c any idea of why this might be the case?

@dbsousa01 Not sure I understand what you did – did you `save_pretrained()` the model from version `2.11.0` of the library?

@julien-c yes, saved the model locally using `save_pretrained` from the version `2.11.0` and then updated the package to the latest version, `3.1.0`, then used the method `from_pretrained` to load it up again, from the local path. By doing this, the warning no longer shows (which could be a bug) **but**, and this is why it's interesting, the model starts outputting stable results, which didn't happen before when I downloaded the model from the latest version