AutoModel doesn't work with DPRContextEncoder
Author: antoniolanza1996
Pull request: no
Labels: name: wontfix

## Environment info
- `transformers` version: 4.2.2
- Platform: Darwin-19.6.0-x86_64-i386-64bit
- Python version: 3.7.9
- PyTorch version (GPU?): 1.7.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
@lhoestq 
@patrickvonplaten 

## Information

If I run:
```python
model = AutoModel.from_pretrained('facebook/dpr-ctx_encoder-single-nq-base')
```

AutoModel infers `DPRQuestionEncoder` but not the correct class (i.e. `DPRContextEncoder`). Thus we can't use the correct model weights. 

The output is:
```
Some weights of the model checkpoint at facebook/dpr-ctx_encoder-single-nq-base were not used when initializing DPRQuestionEncoder: ['ctx_encoder.bert_model.embeddings.word_embeddings.weight', 'ctx_encoder.bert_model.embeddings.position_embeddings.weight', 'ctx_encoder.bert_model.embeddings.token_type_embeddings.weight', 'ctx_encoder.bert_model.embeddings.LayerNorm.weight', 'ctx_encoder.bert_model.embeddings.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.0.attention.self.query.weight', 'ctx_encoder.bert_model.encoder.layer.0.attention.self.query.bias', 'ctx_encoder.bert_model.encoder.layer.0.attention.self.key.weight', 'ctx_encoder.bert_model.encoder.layer.0.attention.self.key.bias', 'ctx_encoder.bert_model.encoder.layer.0.attention.self.value.weight', 'ctx_encoder.bert_model.encoder.layer.0.attention.self.value.bias', 'ctx_encoder.bert_model.encoder.layer.0.attention.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.0.attention.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.0.attention.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.0.attention.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.0.intermediate.dense.weight', 'ctx_encoder.bert_model.encoder.layer.0.intermediate.dense.bias', 'ctx_encoder.bert_model.encoder.layer.0.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.0.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.0.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.0.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.1.attention.self.query.weight', 'ctx_encoder.bert_model.encoder.layer.1.attention.self.query.bias', 'ctx_encoder.bert_model.encoder.layer.1.attention.self.key.weight', 'ctx_encoder.bert_model.encoder.layer.1.attention.self.key.bias', 'ctx_encoder.bert_model.encoder.layer.1.attention.self.value.weight', 'ctx_encoder.bert_model.encoder.layer.1.attention.self.value.bias', 'ctx_encoder.bert_model.encoder.layer.1.attention.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.1.attention.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.1.attention.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.1.attention.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.1.intermediate.dense.weight', 'ctx_encoder.bert_model.encoder.layer.1.intermediate.dense.bias', 'ctx_encoder.bert_model.encoder.layer.1.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.1.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.1.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.1.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.2.attention.self.query.weight', 'ctx_encoder.bert_model.encoder.layer.2.attention.self.query.bias', 'ctx_encoder.bert_model.encoder.layer.2.attention.self.key.weight', 'ctx_encoder.bert_model.encoder.layer.2.attention.self.key.bias', 'ctx_encoder.bert_model.encoder.layer.2.attention.self.value.weight', 'ctx_encoder.bert_model.encoder.layer.2.attention.self.value.bias', 'ctx_encoder.bert_model.encoder.layer.2.attention.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.2.attention.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.2.attention.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.2.attention.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.2.intermediate.dense.weight', 'ctx_encoder.bert_model.encoder.layer.2.intermediate.dense.bias', 'ctx_encoder.bert_model.encoder.layer.2.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.2.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.2.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.2.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.3.attention.self.query.weight', 'ctx_encoder.bert_model.encoder.layer.3.attention.self.query.bias', 'ctx_encoder.bert_model.encoder.layer.3.attention.self.key.weight', 'ctx_encoder.bert_model.encoder.layer.3.attention.self.key.bias', 'ctx_encoder.bert_model.encoder.layer.3.attention.self.value.weight', 'ctx_encoder.bert_model.encoder.layer.3.attention.self.value.bias', 'ctx_encoder.bert_model.encoder.layer.3.attention.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.3.attention.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.3.attention.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.3.attention.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.3.intermediate.dense.weight', 'ctx_encoder.bert_model.encoder.layer.3.intermediate.dense.bias', 'ctx_encoder.bert_model.encoder.layer.3.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.3.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.3.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.3.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.4.attention.self.query.weight', 'ctx_encoder.bert_model.encoder.layer.4.attention.self.query.bias', 'ctx_encoder.bert_model.encoder.layer.4.attention.self.key.weight', 'ctx_encoder.bert_model.encoder.layer.4.attention.self.key.bias', 'ctx_encoder.bert_model.encoder.layer.4.attention.self.value.weight', 'ctx_encoder.bert_model.encoder.layer.4.attention.self.value.bias', 'ctx_encoder.bert_model.encoder.layer.4.attention.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.4.attention.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.4.attention.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.4.attention.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.4.intermediate.dense.weight', 'ctx_encoder.bert_model.encoder.layer.4.intermediate.dense.bias', 'ctx_encoder.bert_model.encoder.layer.4.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.4.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.4.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.4.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.5.attention.self.query.weight', 'ctx_encoder.bert_model.encoder.layer.5.attention.self.query.bias', 'ctx_encoder.bert_model.encoder.layer.5.attention.self.key.weight', 'ctx_encoder.bert_model.encoder.layer.5.attention.self.key.bias', 'ctx_encoder.bert_model.encoder.layer.5.attention.self.value.weight', 'ctx_encoder.bert_model.encoder.layer.5.attention.self.value.bias', 'ctx_encoder.bert_model.encoder.layer.5.attention.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.5.attention.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.5.attention.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.5.attention.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.5.intermediate.dense.weight', 'ctx_encoder.bert_model.encoder.layer.5.intermediate.dense.bias', 'ctx_encoder.bert_model.encoder.layer.5.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.5.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.5.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.5.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.6.attention.self.query.weight', 'ctx_encoder.bert_model.encoder.layer.6.attention.self.query.bias', 'ctx_encoder.bert_model.encoder.layer.6.attention.self.key.weight', 'ctx_encoder.bert_model.encoder.layer.6.attention.self.key.bias', 'ctx_encoder.bert_model.encoder.layer.6.attention.self.value.weight', 'ctx_encoder.bert_model.encoder.layer.6.attention.self.value.bias', 'ctx_encoder.bert_model.encoder.layer.6.attention.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.6.attention.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.6.attention.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.6.attention.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.6.intermediate.dense.weight', 'ctx_encoder.bert_model.encoder.layer.6.intermediate.dense.bias', 'ctx_encoder.bert_model.encoder.layer.6.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.6.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.6.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.6.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.7.attention.self.query.weight', 'ctx_encoder.bert_model.encoder.layer.7.attention.self.query.bias', 'ctx_encoder.bert_model.encoder.layer.7.attention.self.key.weight', 'ctx_encoder.bert_model.encoder.layer.7.attention.self.key.bias', 'ctx_encoder.bert_model.encoder.layer.7.attention.self.value.weight', 'ctx_encoder.bert_model.encoder.layer.7.attention.self.value.bias', 'ctx_encoder.bert_model.encoder.layer.7.attention.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.7.attention.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.7.attention.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.7.attention.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.7.intermediate.dense.weight', 'ctx_encoder.bert_model.encoder.layer.7.intermediate.dense.bias', 'ctx_encoder.bert_model.encoder.layer.7.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.7.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.7.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.7.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.8.attention.self.query.weight', 'ctx_encoder.bert_model.encoder.layer.8.attention.self.query.bias', 'ctx_encoder.bert_model.encoder.layer.8.attention.self.key.weight', 'ctx_encoder.bert_model.encoder.layer.8.attention.self.key.bias', 'ctx_encoder.bert_model.encoder.layer.8.attention.self.value.weight', 'ctx_encoder.bert_model.encoder.layer.8.attention.self.value.bias', 'ctx_encoder.bert_model.encoder.layer.8.attention.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.8.attention.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.8.attention.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.8.attention.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.8.intermediate.dense.weight', 'ctx_encoder.bert_model.encoder.layer.8.intermediate.dense.bias', 'ctx_encoder.bert_model.encoder.layer.8.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.8.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.8.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.8.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.9.attention.self.query.weight', 'ctx_encoder.bert_model.encoder.layer.9.attention.self.query.bias', 'ctx_encoder.bert_model.encoder.layer.9.attention.self.key.weight', 'ctx_encoder.bert_model.encoder.layer.9.attention.self.key.bias', 'ctx_encoder.bert_model.encoder.layer.9.attention.self.value.weight', 'ctx_encoder.bert_model.encoder.layer.9.attention.self.value.bias', 'ctx_encoder.bert_model.encoder.layer.9.attention.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.9.attention.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.9.attention.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.9.attention.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.9.intermediate.dense.weight', 'ctx_encoder.bert_model.encoder.layer.9.intermediate.dense.bias', 'ctx_encoder.bert_model.encoder.layer.9.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.9.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.9.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.9.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.10.attention.self.query.weight', 'ctx_encoder.bert_model.encoder.layer.10.attention.self.query.bias', 'ctx_encoder.bert_model.encoder.layer.10.attention.self.key.weight', 'ctx_encoder.bert_model.encoder.layer.10.attention.self.key.bias', 'ctx_encoder.bert_model.encoder.layer.10.attention.self.value.weight', 'ctx_encoder.bert_model.encoder.layer.10.attention.self.value.bias', 'ctx_encoder.bert_model.encoder.layer.10.attention.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.10.attention.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.10.attention.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.10.attention.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.10.intermediate.dense.weight', 'ctx_encoder.bert_model.encoder.layer.10.intermediate.dense.bias', 'ctx_encoder.bert_model.encoder.layer.10.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.10.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.10.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.10.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.11.attention.self.query.weight', 'ctx_encoder.bert_model.encoder.layer.11.attention.self.query.bias', 'ctx_encoder.bert_model.encoder.layer.11.attention.self.key.weight', 'ctx_encoder.bert_model.encoder.layer.11.attention.self.key.bias', 'ctx_encoder.bert_model.encoder.layer.11.attention.self.value.weight', 'ctx_encoder.bert_model.encoder.layer.11.attention.self.value.bias', 'ctx_encoder.bert_model.encoder.layer.11.attention.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.11.attention.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.11.attention.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.11.attention.output.LayerNorm.bias', 'ctx_encoder.bert_model.encoder.layer.11.intermediate.dense.weight', 'ctx_encoder.bert_model.encoder.layer.11.intermediate.dense.bias', 'ctx_encoder.bert_model.encoder.layer.11.output.dense.weight', 'ctx_encoder.bert_model.encoder.layer.11.output.dense.bias', 'ctx_encoder.bert_model.encoder.layer.11.output.LayerNorm.weight', 'ctx_encoder.bert_model.encoder.layer.11.output.LayerNorm.bias', 'ctx_encoder.bert_model.pooler.dense.weight', 'ctx_encoder.bert_model.pooler.dense.bias']
- This IS expected if you are initializing DPRQuestionEncoder from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing DPRQuestionEncoder from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of DPRQuestionEncoder were not initialized from the model checkpoint at facebook/dpr-ctx_encoder-single-nq-base and are newly initialized: ['bert_model.embeddings.word_embeddings.weight', 'bert_model.embeddings.position_embeddings.weight', 'bert_model.embeddings.token_type_embeddings.weight', 'bert_model.embeddings.LayerNorm.weight', 'bert_model.embeddings.LayerNorm.bias', 'bert_model.encoder.layer.0.attention.self.query.weight', 'bert_model.encoder.layer.0.attention.self.query.bias', 'bert_model.encoder.layer.0.attention.self.key.weight', 'bert_model.encoder.layer.0.attention.self.key.bias', 'bert_model.encoder.layer.0.attention.self.value.weight', 'bert_model.encoder.layer.0.attention.self.value.bias', 'bert_model.encoder.layer.0.attention.output.dense.weight', 'bert_model.encoder.layer.0.attention.output.dense.bias', 'bert_model.encoder.layer.0.attention.output.LayerNorm.weight', 'bert_model.encoder.layer.0.attention.output.LayerNorm.bias', 'bert_model.encoder.layer.0.intermediate.dense.weight', 'bert_model.encoder.layer.0.intermediate.dense.bias', 'bert_model.encoder.layer.0.output.dense.weight', 'bert_model.encoder.layer.0.output.dense.bias', 'bert_model.encoder.layer.0.output.LayerNorm.weight', 'bert_model.encoder.layer.0.output.LayerNorm.bias', 'bert_model.encoder.layer.1.attention.self.query.weight', 'bert_model.encoder.layer.1.attention.self.query.bias', 'bert_model.encoder.layer.1.attention.self.key.weight', 'bert_model.encoder.layer.1.attention.self.key.bias', 'bert_model.encoder.layer.1.attention.self.value.weight', 'bert_model.encoder.layer.1.attention.self.value.bias', 'bert_model.encoder.layer.1.attention.output.dense.weight', 'bert_model.encoder.layer.1.attention.output.dense.bias', 'bert_model.encoder.layer.1.attention.output.LayerNorm.weight', 'bert_model.encoder.layer.1.attention.output.LayerNorm.bias', 'bert_model.encoder.layer.1.intermediate.dense.weight', 'bert_model.encoder.layer.1.intermediate.dense.bias', 'bert_model.encoder.layer.1.output.dense.weight', 'bert_model.encoder.layer.1.output.dense.bias', 'bert_model.encoder.layer.1.output.LayerNorm.weight', 'bert_model.encoder.layer.1.output.LayerNorm.bias', 'bert_model.encoder.layer.2.attention.self.query.weight', 'bert_model.encoder.layer.2.attention.self.query.bias', 'bert_model.encoder.layer.2.attention.self.key.weight', 'bert_model.encoder.layer.2.attention.self.key.bias', 'bert_model.encoder.layer.2.attention.self.value.weight', 'bert_model.encoder.layer.2.attention.self.value.bias', 'bert_model.encoder.layer.2.attention.output.dense.weight', 'bert_model.encoder.layer.2.attention.output.dense.bias', 'bert_model.encoder.layer.2.attention.output.LayerNorm.weight', 'bert_model.encoder.layer.2.attention.output.LayerNorm.bias', 'bert_model.encoder.layer.2.intermediate.dense.weight', 'bert_model.encoder.layer.2.intermediate.dense.bias', 'bert_model.encoder.layer.2.output.dense.weight', 'bert_model.encoder.layer.2.output.dense.bias', 'bert_model.encoder.layer.2.output.LayerNorm.weight', 'bert_model.encoder.layer.2.output.LayerNorm.bias', 'bert_model.encoder.layer.3.attention.self.query.weight', 'bert_model.encoder.layer.3.attention.self.query.bias', 'bert_model.encoder.layer.3.attention.self.key.weight', 'bert_model.encoder.layer.3.attention.self.key.bias', 'bert_model.encoder.layer.3.attention.self.value.weight', 'bert_model.encoder.layer.3.attention.self.value.bias', 'bert_model.encoder.layer.3.attention.output.dense.weight', 'bert_model.encoder.layer.3.attention.output.dense.bias', 'bert_model.encoder.layer.3.attention.output.LayerNorm.weight', 'bert_model.encoder.layer.3.attention.output.LayerNorm.bias', 'bert_model.encoder.layer.3.intermediate.dense.weight', 'bert_model.encoder.layer.3.intermediate.dense.bias', 'bert_model.encoder.layer.3.output.dense.weight', 'bert_model.encoder.layer.3.output.dense.bias', 'bert_model.encoder.layer.3.output.LayerNorm.weight', 'bert_model.encoder.layer.3.output.LayerNorm.bias', 'bert_model.encoder.layer.4.attention.self.query.weight', 'bert_model.encoder.layer.4.attention.self.query.bias', 'bert_model.encoder.layer.4.attention.self.key.weight', 'bert_model.encoder.layer.4.attention.self.key.bias', 'bert_model.encoder.layer.4.attention.self.value.weight', 'bert_model.encoder.layer.4.attention.self.value.bias', 'bert_model.encoder.layer.4.attention.output.dense.weight', 'bert_model.encoder.layer.4.attention.output.dense.bias', 'bert_model.encoder.layer.4.attention.output.LayerNorm.weight', 'bert_model.encoder.layer.4.attention.output.LayerNorm.bias', 'bert_model.encoder.layer.4.intermediate.dense.weight', 'bert_model.encoder.layer.4.intermediate.dense.bias', 'bert_model.encoder.layer.4.output.dense.weight', 'bert_model.encoder.layer.4.output.dense.bias', 'bert_model.encoder.layer.4.output.LayerNorm.weight', 'bert_model.encoder.layer.4.output.LayerNorm.bias', 'bert_model.encoder.layer.5.attention.self.query.weight', 'bert_model.encoder.layer.5.attention.self.query.bias', 'bert_model.encoder.layer.5.attention.self.key.weight', 'bert_model.encoder.layer.5.attention.self.key.bias', 'bert_model.encoder.layer.5.attention.self.value.weight', 'bert_model.encoder.layer.5.attention.self.value.bias', 'bert_model.encoder.layer.5.attention.output.dense.weight', 'bert_model.encoder.layer.5.attention.output.dense.bias', 'bert_model.encoder.layer.5.attention.output.LayerNorm.weight', 'bert_model.encoder.layer.5.attention.output.LayerNorm.bias', 'bert_model.encoder.layer.5.intermediate.dense.weight', 'bert_model.encoder.layer.5.intermediate.dense.bias', 'bert_model.encoder.layer.5.output.dense.weight', 'bert_model.encoder.layer.5.output.dense.bias', 'bert_model.encoder.layer.5.output.LayerNorm.weight', 'bert_model.encoder.layer.5.output.LayerNorm.bias', 'bert_model.encoder.layer.6.attention.self.query.weight', 'bert_model.encoder.layer.6.attention.self.query.bias', 'bert_model.encoder.layer.6.attention.self.key.weight', 'bert_model.encoder.layer.6.attention.self.key.bias', 'bert_model.encoder.layer.6.attention.self.value.weight', 'bert_model.encoder.layer.6.attention.self.value.bias', 'bert_model.encoder.layer.6.attention.output.dense.weight', 'bert_model.encoder.layer.6.attention.output.dense.bias', 'bert_model.encoder.layer.6.attention.output.LayerNorm.weight', 'bert_model.encoder.layer.6.attention.output.LayerNorm.bias', 'bert_model.encoder.layer.6.intermediate.dense.weight', 'bert_model.encoder.layer.6.intermediate.dense.bias', 'bert_model.encoder.layer.6.output.dense.weight', 'bert_model.encoder.layer.6.output.dense.bias', 'bert_model.encoder.layer.6.output.LayerNorm.weight', 'bert_model.encoder.layer.6.output.LayerNorm.bias', 'bert_model.encoder.layer.7.attention.self.query.weight', 'bert_model.encoder.layer.7.attention.self.query.bias', 'bert_model.encoder.layer.7.attention.self.key.weight', 'bert_model.encoder.layer.7.attention.self.key.bias', 'bert_model.encoder.layer.7.attention.self.value.weight', 'bert_model.encoder.layer.7.attention.self.value.bias', 'bert_model.encoder.layer.7.attention.output.dense.weight', 'bert_model.encoder.layer.7.attention.output.dense.bias', 'bert_model.encoder.layer.7.attention.output.LayerNorm.weight', 'bert_model.encoder.layer.7.attention.output.LayerNorm.bias', 'bert_model.encoder.layer.7.intermediate.dense.weight', 'bert_model.encoder.layer.7.intermediate.dense.bias', 'bert_model.encoder.layer.7.output.dense.weight', 'bert_model.encoder.layer.7.output.dense.bias', 'bert_model.encoder.layer.7.output.LayerNorm.weight', 'bert_model.encoder.layer.7.output.LayerNorm.bias', 'bert_model.encoder.layer.8.attention.self.query.weight', 'bert_model.encoder.layer.8.attention.self.query.bias', 'bert_model.encoder.layer.8.attention.self.key.weight', 'bert_model.encoder.layer.8.attention.self.key.bias', 'bert_model.encoder.layer.8.attention.self.value.weight', 'bert_model.encoder.layer.8.attention.self.value.bias', 'bert_model.encoder.layer.8.attention.output.dense.weight', 'bert_model.encoder.layer.8.attention.output.dense.bias', 'bert_model.encoder.layer.8.attention.output.LayerNorm.weight', 'bert_model.encoder.layer.8.attention.output.LayerNorm.bias', 'bert_model.encoder.layer.8.intermediate.dense.weight', 'bert_model.encoder.layer.8.intermediate.dense.bias', 'bert_model.encoder.layer.8.output.dense.weight', 'bert_model.encoder.layer.8.output.dense.bias', 'bert_model.encoder.layer.8.output.LayerNorm.weight', 'bert_model.encoder.layer.8.output.LayerNorm.bias', 'bert_model.encoder.layer.9.attention.self.query.weight', 'bert_model.encoder.layer.9.attention.self.query.bias', 'bert_model.encoder.layer.9.attention.self.key.weight', 'bert_model.encoder.layer.9.attention.self.key.bias', 'bert_model.encoder.layer.9.attention.self.value.weight', 'bert_model.encoder.layer.9.attention.self.value.bias', 'bert_model.encoder.layer.9.attention.output.dense.weight', 'bert_model.encoder.layer.9.attention.output.dense.bias', 'bert_model.encoder.layer.9.attention.output.LayerNorm.weight', 'bert_model.encoder.layer.9.attention.output.LayerNorm.bias', 'bert_model.encoder.layer.9.intermediate.dense.weight', 'bert_model.encoder.layer.9.intermediate.dense.bias', 'bert_model.encoder.layer.9.output.dense.weight', 'bert_model.encoder.layer.9.output.dense.bias', 'bert_model.encoder.layer.9.output.LayerNorm.weight', 'bert_model.encoder.layer.9.output.LayerNorm.bias', 'bert_model.encoder.layer.10.attention.self.query.weight', 'bert_model.encoder.layer.10.attention.self.query.bias', 'bert_model.encoder.layer.10.attention.self.key.weight', 'bert_model.encoder.layer.10.attention.self.key.bias', 'bert_model.encoder.layer.10.attention.self.value.weight', 'bert_model.encoder.layer.10.attention.self.value.bias', 'bert_model.encoder.layer.10.attention.output.dense.weight', 'bert_model.encoder.layer.10.attention.output.dense.bias', 'bert_model.encoder.layer.10.attention.output.LayerNorm.weight', 'bert_model.encoder.layer.10.attention.output.LayerNorm.bias', 'bert_model.encoder.layer.10.intermediate.dense.weight', 'bert_model.encoder.layer.10.intermediate.dense.bias', 'bert_model.encoder.layer.10.output.dense.weight', 'bert_model.encoder.layer.10.output.dense.bias', 'bert_model.encoder.layer.10.output.LayerNorm.weight', 'bert_model.encoder.layer.10.output.LayerNorm.bias', 'bert_model.encoder.layer.11.attention.self.query.weight', 'bert_model.encoder.layer.11.attention.self.query.bias', 'bert_model.encoder.layer.11.attention.self.key.weight', 'bert_model.encoder.layer.11.attention.self.key.bias', 'bert_model.encoder.layer.11.attention.self.value.weight', 'bert_model.encoder.layer.11.attention.self.value.bias', 'bert_model.encoder.layer.11.attention.output.dense.weight', 'bert_model.encoder.layer.11.attention.output.dense.bias', 'bert_model.encoder.layer.11.attention.output.LayerNorm.weight', 'bert_model.encoder.layer.11.attention.output.LayerNorm.bias', 'bert_model.encoder.layer.11.intermediate.dense.weight', 'bert_model.encoder.layer.11.intermediate.dense.bias', 'bert_model.encoder.layer.11.output.dense.weight', 'bert_model.encoder.layer.11.output.dense.bias', 'bert_model.encoder.layer.11.output.LayerNorm.weight', 'bert_model.encoder.layer.11.output.LayerNorm.bias', 'bert_model.pooler.dense.weight', 'bert_model.pooler.dense.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
```

I think could be useful to generalise this behaviour and automatically detect whether the model is a context/document encoder or question encoder. 

Comments:
This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.

I ran into this recently and posted a question about it [on the forum](https://discuss.huggingface.co/t/dpr-pretrained-context-encoder-unused-weight-warning/11265), awaiting response.

I don't have a solution but, as a workaround, if you use `DPRContextEncoder` instead of `AutoModel`,
```
from transformers import DPRContextEncoder
context_encoder = DPRContextEncoder.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base")
```
you don't get the runtime warning.