transformers vs pytorch_pretrained_bert giving different scores for BertForNextSentencePrediction
Author: AjitAntony
Pull request: no

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
import torch
from transformers import BertTokenizer, BertModel, BertForMaskedLM,BertForNextSentencePrediction
tokenizer=BertTokenizer.from_pretrained('bert-base-uncased')
BertNSP=BertForNextSentencePrediction.from_pretrained('bert-base-uncased')

text1 = "How old are you?"
text2 = "The Eiffel Tower is in Paris"

text1_toks = ["[CLS]"] + tokenizer.tokenize(text1) + ["[SEP]"]
text2_toks = tokenizer.tokenize(text2) + ["[SEP]"]
text=text1_toks+text2_toks
indexed_tokens = tokenizer.convert_tokens_to_ids(text)
segments_ids = [0]*len(text1_toks) + [1]*len(text2_toks)

tokens_tensor = torch.tensor([indexed_tokens])
segments_tensors = torch.tensor([segments_ids])
BertNSP.eval()
prediction = BertNSP(tokens_tensor, segments_tensors)
prediction=prediction[0] # tuple to tensor
print(predictions)
softmax = torch.nn.Softmax(dim=1)
prediction_sm = softmax(prediction)
print (prediction_sm)

output:
tensor([[ 2.1772, -0.8097]], grad_fn=)
tensor([[0.9923, 0.0077]], grad_fn=)

Now when trying with pytorch_pretrained_bert :

!pip install pytorch_pretrained_bert
import torch
import pytorch_pretrained_bert
from pytorch_pretrained_bert import BertTokenizer, BertAdam, BertForNextSentencePrediction

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForNextSentencePrediction.from_pretrained('bert-base-uncased')

text1 = "How old are you?"
text2 = "The Eiffel Tower is in Paris"

text1_toks = ["[CLS]"] + tokenizer.tokenize(text1) + ["[SEP]"]
text2_toks = tokenizer.tokenize(text2) + ["[SEP]"]
text=text1_toks+text2_toks
print(text)
indexed_tokens = tokenizer.convert_tokens_to_ids(text)
segments_ids = [0]*len(text1_toks) + [1]*len(text2_toks)

tokens_tensor = torch.tensor([indexed_tokens])
segments_tensors = torch.tensor([segments_ids])
print(indexed_tokens)
print(segments_ids)

model.eval()
prediction = model(tokens_tensor, segments_tensors)

print(prediction)
softmax = torch.nn.Softmax(dim=1)
prediction_sm = softmax(prediction)
print (prediction_sm)

output:
tensor([[-2.3808, 5.4018]], grad_fn=)
tensor([[4.1673e-04, 9.9958e-01]], grad_fn=)

1. which is the correct score here ? is the softmax score from transformers  model correct or pytorch_pretrained_bert  model correct ? 

2.Also the output of model in pytorch_pretrained_bert is tensor but output of model in transformers is tuple .Why again is it like this ?

Comments:
Hi, since `pytorch_pretrained_BERT`, many breaking changes have happened, two of which are causing confusion in your snippet:

- The order of arguments in the forward call has been slightly changed [(v2.0.0)](https://github.com/huggingface/transformers/releases/tag/v2.0.0)
- The models now always return tuples [(v1.0.0)](https://github.com/huggingface/transformers/releases/tag/v1.0.0).

The attention mask and token type ids order has been changed for the forward method to better respect the order of importance, which is important when compiling a model with torchscript.

Update the forward call of your model as such to obtain identical results on both:

```py
prediction = model(tokens_tensor, token_type_ids=segments_tensors)
```

To prevent further breaking changes from affecting your workflow, we recommend using named arguments when calling different methods, like it is done in the aforementioned snippet.

@LysandreJik thanks for the information .Dose this apply for all the bert models or only for the next sentence prediction alone ?

I applies to all models.