Predict Mode: Weights of BertForQuestionAnswering not initialized from pretrained model
Author: ghost
Pull request: no

I am running following in just prediction mode:
`(berttorch363) sandeepbhutani304@pytorch-bert-2:~/pytorch-pretrained-BERT/examples$ python run_squad.py   --bert_model bert-large-uncased    --do_predict   --do_lower_case    --predict_file $SQUAD_DIR/dev-v1.1_sand.json   --train_batch_size 12   --learning_rate 3e-5   --num_train_epochs 2.0   --max_seq_length 384   --doc_stride 128   --output_dir debug_squad9`

Every time I am getting  following **not initialized message** (**Answers are also wrong**, both with bert-base and bert-large). Is something wrong going on ? (I am running prediction only on CPU)

```
01/03/2019 13:43:09 - INFO - pytorch_pretrained_bert.modeling -   Weights of BertForQuestionAnswering not initialized from pretrained model: ['qa_outputs.weight', 'qa_outputs.bias']
01/03/2019 13:43:09 - INFO - pytorch_pretrained_bert.modeling -   Weights from pretrained model not used in BertForQuestionAnswering: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias']
```


Complete log (if interested) is below:

```
(berttorch363) sandeepbhutani304@pytorch-bert-2:~/pytorch-pretrained-BERT/examples$ python run_squad.py   --bert_model bert-base-uncased    --do_predict   --do_lower_case    --predict_file $SQUAD_DIR/dev-v1.1_sand.json   --train_batch_size 12   --learning_rate 3e-5   --num_train_epochs 2.0   --max_seq_length 384   --doc_stride 128   --output_dir debug_squad09
Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex.
/opt/anaconda3/envs/berttorch363/lib/python3.6/site-packages/pytorch_pretrained_bert/modeling.py
01/03/2019 13:50:06 - INFO - __main__ -   device: cpu n_gpu: 0, distributed training: False, 16-bits training: False
01/03/2019 13:50:07 - INFO - pytorch_pretrained_bert.tokenization -   loading vocabulary file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-vocab.txt from cache at /home/sandeepbhutani304/.pytorch_pretrained_bert/26bc1ad6c0ac742e9b52263248f6d0f00068293b33709fae12320c0e35ccfbbb.542ce4285a40d23a559526243235df47c5f75c197f04f37d1a0c124c32c9a084
01/03/2019 13:50:08 - INFO - pytorch_pretrained_bert.modeling -   loading archive file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased.tar.gz from cache at /home/sandeepbhutani304/.pytorch_pretrained_bert/distributed_-1/9c41111e2de84547a463fd39217199738d1e3deb72d4fec4399e6e241983c6f0.ae3cef932725ca7a30cdcb93fc6e09150a55e2a130ec7af63975a16c153ae2ba
01/03/2019 13:50:08 - INFO - pytorch_pretrained_bert.modeling -   extracting archive file /home/sandeepbhutani304/.pytorch_pretrained_bert/distributed_-1/9c41111e2de84547a463fd39217199738d1e3deb72d4fec4399e6e241983c6f0.ae3cef932725ca7a30cdcb93fc6e09150a55e2a130ec7af63975a16c153ae2ba to temp dir /tmp/tmpm3rr0ye3
01/03/2019 13:50:14 - INFO - pytorch_pretrained_bert.modeling -   Model config {
  "attention_probs_dropout_prob": 0.1,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "max_position_embeddings": 512,
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "type_vocab_size": 2,
  "vocab_size": 30522
}
01/03/2019 13:50:18 - INFO - pytorch_pretrained_bert.modeling -   Weights from pretrained model not used in BertForQuestionAnswering: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias']
01/03/2019 13:50:20 - INFO - pytorch_pretrained_bert.modeling -   loading archive file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased.tar.gz from cache at /home/sandeepbhutani304/.pytorch_pretrained_bert/9c41111e2de84547a463fd39217199738d1e3deb72d4fec4399e6e241983c6f0.ae3cef932725ca7a30cdcb93fc6e09150a55e2a130ec7af63975a16c153ae2ba
01/03/2019 13:50:20 - INFO - pytorch_pretrained_bert.modeling -   extracting archive file /home/sandeepbhutani304/.pytorch_pretrained_bert/9c41111e2de84547a463fd39217199738d1e3deb72d4fec4399e6e241983c6f0.ae3cef932725ca7a30cdcb93fc6e09150a55e2a130ec7af63975a16c153ae2ba to temp dir /tmp/tmpt86p1sz4
01/03/2019 13:50:32 - INFO - pytorch_pretrained_bert.modeling -   Model config {
  "attention_probs_dropout_prob": 0.1,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "max_position_embeddings": 512,
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "type_vocab_size": 2,
  "vocab_size": 30522
}

01/03/2019 13:50:35 - INFO - __main__ -   *** Example ***
01/03/2019 13:50:35 - INFO - __main__ -   unique_id: 1000000000
01/03/2019 13:50:35 - INFO - __main__ -   example_index: 0
01/03/2019 13:50:35 - INFO - __main__ -   doc_span_index: 0
ce , a hare saw a tor ##to ##ise walking slowly with a heavy shell on his back . the hare was very proud of himself and he asked the tor ##to ##ise . ' shall we have a race ? ' the tor ##to ##ise agreed . they started the running race . the hare ran very fast . but the tor ##to ##ise walked very slowly . the proud hair rested under a tree and soon slept off . but the tor ##to ##ise walked very fast , slowly and steadily and reached the goal . at last , the tor ##to ##ise won the race . moral : pride goes before a fall . [SEP]
01/03/2019 13:50:35 - INFO - __main__ -   token_to_orig_map: 
--- ommited log ---
01/03/2019 13:50:35 - INFO - __main__ -   ***** Running predictions *****
01/03/2019 13:50:35 - INFO - __main__ -     Num orig examples = 1
01/03/2019 13:50:35 - INFO - __main__ -     Num split examples = 1
01/03/2019 13:50:35 - INFO - __main__ -     Batch size = 8
01/03/2019 13:50:35 - INFO - __main__ -   Start evaluating
Evaluating:   0%|                                                                                                                 | 0/1 [00:00<?, ?it/s]01/03/2019 13:50:35 - INFO - __main__ -   Processing example: 0
Evaluating: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.26s/it]
01/03/2019 13:50:37 - INFO - __main__ -   Writing predictions to: debug_squad09/predictions.json
01/03/2019 13:50:37 - INFO - __main__ -   Writing nbest to: debug_squad09/nbest_predictions.json
(berttorch363) sandeepbhutani304@pytorch-bert-2:~/pytorch-pretrained-BERT/examples$ 
````

Comments:
@thomwolf Can you please reply here. This issue is different from issue#160
#160 is for training mode, this issue is for prediction mode. ( I hope prediction can run on CPU)

Those messages are correct, the pretrained weights that have been released by Google Brain are just the ones of the core network. They did not release task specific (such as SQuAD) weights. To get a model that solves this task, you would have to train one yourself or get it from someone else.

To answer your second question, yes, predictions can run on CPU.

@rodgzilla is right (even though I think prediction on CPU will be very slow, you should use a GPU)

Hi @rodgzilla and @thomwolf Thanks for reply. I figured out after reply that in CPU, GPU talks, I forgot to train the model with squad training data, that is why those warning messages were coming.  
  
training on 1 GPU took 2.5 hours per epoch (I had to reduce the max_seq_length due to large memory consumption). And after training prediction is running fine on CPU.  
  
Would you like to comment on below 2 observations:  
1. pytorch version is faster than tensorflow version. Good but why.
2. Answers of tensorflow version and pytorch version are different, even though the training, question and data title is same.


> Those messages are correct, the pretrained weights that have been released by Google Brain are just the ones of the core network. They did not release task-specific (such as SQuAD) weights. To get a model that solves this task, you would have to train one yourself or get it from someone else.

Does it mean that training-phase will not train BERT transformer parameters? If BERT params are tuned during the training phase, it should be stored in the output model. During the prediction time, tuned params should be used instead of loading BERT params from the original file.