behaviour of ZeroShotClassification using facebook/bart-large-mnli is different on online demo vs local machine
Author: turmeric-blend
Pull request: no

## Environment info
     
- `transformers` version: 3.4.0
- Platform: Ubuntu 20.04
- Python version: 3.7.7
- PyTorch version (GPU?): 1.6.0 (GPU:Yes)
- Tensorflow version (GPU?): No
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 blenderbot: @mariamabarham
 Bart: @sshleifer
 Marian: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

@sshleifer

## Information

Model I am using (Bert, XLNet ...): facebook/bart-large-mnli

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

First I tried the hosted demo online at huggingface, which gives me a very high score of **0.99 for travelling (as expected)**:
![Screenshot from 2020-10-29 00-02-40](https://user-images.githubusercontent.com/62788745/97464214-949e3480-197b-11eb-9fc8-72d5fcdcc49c.png)

Then I tried to run the code on my local machine, which returns **very different scores for all labels** (poor scores):

```
from transformers import pipeline
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-mnli")
model = AutoModel.from_pretrained("facebook/bart-large-mnli")

zsc = pipeline(task='zero-shot-classification', tokenizer=tokenizer, model=model)

sequences = 'one day I will see the world'
candidate_labels = ['travelling', 'cooking', 'dancing']

results = zsc(sequences=sequences, candidate_labels=candidate_labels, multi_class=False)
print(results)
>>>{'sequence': 'one day I will see the world',
'labels': ['travelling', 'dancing', 'cooking'],
'scores': [0.5285395979881287, 0.2499372661113739, 0.22152313590049744]}
```

I **got this warning message when initializing the model**:
`model = AutoModel.from_pretrained("facebook/bart-large-mnli")`

```
Some weights of the model checkpoint at facebook/bart-large-mnli were not used when initializing BartModel: ['model.encoder.version', 'model.decoder.version']
- This IS expected if you are initializing BartModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing BartModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
```

## Expected behavior

Code on my local machine's **_score_** to be quite similar to the online demo.


Comments:
Replace `AutoModel` with `AutoModelForSequenceClassification`. The former won't add the sequence classification head, i.e. it will use `BartModel` instead of `BartForSequenceClassification`, so the pipeline is trying to use just the outputs of the encoder instead of the NLI predictions in your snippet.

@joeddav  that fixed it thanks !

Have the same problem:

conda environment: Python 3.7.9
```
pip3 install torch==1.6
pip3 install transformers
```

Running

```
from transformers import AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained("facebook/bart-large-mnli")
```

Results in message:

> Some weights of the model checkpoint at facebook/bart-large-mnli were not used when initializing BartForSequenceClassification: ['model.encoder.version', 'model.decoder.version']
> - This IS expected if you are initializing BartForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
> - This IS NOT expected if you are initializing BartForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

@turmeric-blend: How is my setup different from yours? 


actually the error message was still there after the fix, but the scores running on local machine were consistent with the online demo @gustavengstrom 

any ideas why is there still the warning message @joeddav ?

Yeah that warning isn't a concern. It's just letting you know that some of the parameters checkpointed in the pretrained model were not able to be matched with the model class, but in this case it's just a couple of meta-fields (encoder/decoder version), so your weights should be matched up fine.