AlbertForSequenceClassification random output
Author: Zjq9409
Pull request: no

I use AlbertForSequenceClassification interface as follows:
`import torch
from transformers import  BertTokenizer,AlbertConfig,AlbertForSequenceClassification
import numpy
pretrained = "./albert_chinese_base"
tokenizer = BertTokenizer.from_pretrained(pretrained)
config = AlbertConfig.from_json_file('./albert_chinese_base/config.json')
config.output_hidden_states = True
model = AlbertForSequenceClassification.from_pretrained(pretrained, config = config)
inputtext = "今天心情情很好啊，买了很多东西，我特别喜欢，终于有了自己喜欢的电子产品，这次总算可以好好学习了"
max_length = 128
tokenized_text=tokenizer.encode_plus(inputtext, add_special_tokens = True, # add [CLS], [SEP]
                                 max_length = max_length, # max length of the text that can go to BERT
                                 pad_to_max_length = True, # add [PAD] tokens
                                 return_attention_mask = True, # add attention mask to not focus on pad tokens
                                 return_tensors="pt")
outputs=model(input_ids=tokenized_text["input_ids"], token_type_ids=tokenized_text["token_type_ids"], attention_mask=tokenized_text["attention_mask"])
print(outputs.logits)`

but, when I run this code, an error occured as follows:

`Some weights of the model checkpoint at ./albert_chinese_base were not used when initializing AlbertForSequenceClassification: ['predictions.bias', 'predictions.LayerNorm.weight', 'predictions.LayerNorm.bias', 'predictions.dense.weight', 'predictions.dense.bias', 'predictions.decoder.weight', 'predictions.decoder.bias']

-This IS expected if you are initializing AlbertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).

-This IS NOT expected if you are initializing AlbertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of AlbertForSequenceClassification were not initialized from the model checkpoint at ./albert_chinese_base and are newly initialized: ['classifier.weight', 'classifier.bias']`

And I print outputs.logits value, found that the value is different every time you run,the logits value like this:
tensor([[0.3077, 0.1200]], grad_fn=)
tensor([[-0.3245, -0.3117]], grad_fn=)
so, I wonder AlbertForSequenceClassification model is not initialized correctly the value of ['classifier.weight', 'classifier.bias'] and
random value every time I run, and BertForSequenceClassification result output is right,so how can I slove the problem?

Comments:
Hello! You're loading a model called `albert_chinese_base`; my guess is that this model only contains the base transformer model and not the sequence classification head that you need.

Does that make sense? You should use a model fine-tuned on sequence classification, and not a base model, if you want to do sequence classification. Of course, that model will be fine-tuned to a specific sequence classification task so it can't be used in any context.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.