pretrained model 
Author: minmummax
Pull request: no

is the pretrained model  downloaded  include word embedding?
I do not see any embedding in your code 
please

Comments:
All the code related to word embeddings is located there https://github.com/huggingface/pytorch-pretrained-BERT/blob/8da280ebbeca5ebd7561fd05af78c65df9161f92/pytorch_pretrained_bert/modeling.py#L172-L200

If you want to access pretrained embeddings, the easier thing to do would be to load a pretrained model and extract its embedding matrices.

> All the code related to word embeddings is located there
> 
> [pytorch-pretrained-BERT/pytorch_pretrained_bert/modeling.py](https://github.com/huggingface/pytorch-pretrained-BERT/blob/8da280ebbeca5ebd7561fd05af78c65df9161f92/pytorch_pretrained_bert/modeling.py#L172-L200)
> 
> Lines 172 to 200 in [8da280e](/huggingface/pytorch-pretrained-BERT/commit/8da280ebbeca5ebd7561fd05af78c65df9161f92)
> 
>  class BertEmbeddings(nn.Module): 
>      """Construct the embeddings from word, position and token_type embeddings. 
>      """ 
>      def __init__(self, config): 
>          super(BertEmbeddings, self).__init__() 
>          self.word_embeddings = nn.Embedding(config.vocab_size, config.hidden_size) 
>          self.position_embeddings = nn.Embedding(config.max_position_embeddings, config.hidden_size) 
>          self.token_type_embeddings = nn.Embedding(config.type_vocab_size, config.hidden_size) 
>   
>          # self.LayerNorm is not snake-cased to stick with TensorFlow model variable name and be able to load 
>          # any TensorFlow checkpoint file 
>          self.LayerNorm = BertLayerNorm(config.hidden_size, eps=1e-12) 
>          self.dropout = nn.Dropout(config.hidden_dropout_prob) 
>   
>      def forward(self, input_ids, token_type_ids=None): 
>          seq_length = input_ids.size(1) 
>          position_ids = torch.arange(seq_length, dtype=torch.long, device=input_ids.device) 
>          position_ids = position_ids.unsqueeze(0).expand_as(input_ids) 
>          if token_type_ids is None: 
>              token_type_ids = torch.zeros_like(input_ids) 
>   
>          words_embeddings = self.word_embeddings(input_ids) 
>          position_embeddings = self.position_embeddings(position_ids) 
>          token_type_embeddings = self.token_type_embeddings(token_type_ids) 
>   
>          embeddings = words_embeddings + position_embeddings + token_type_embeddings 
>          embeddings = self.LayerNorm(embeddings) 
>          embeddings = self.dropout(embeddings) 
>          return embeddings 
> If you want to access pretrained embeddings, the easier thing to do would be to load a pretrained model and extract its embedding matrices.

oh I have seen this code these days . and from this code I think it dose not use the pretrained embedding paras , and what do you mean by load and extract a pretrained model ????  Is it from the original supplies

```python
In [1]: from pytorch_pretrained_bert import BertModel                                                                                                                                    

In [2]: model = BertModel.from_pretrained('bert-base-uncased')                                                                                                                           

In [3]: model.embeddings.word_embeddings                                                                                                                                                 
Out[3]: Embedding(30522, 768)
```

This field of the `BertEmbeddings` class contains the pretrained embeddings. It gets set by calling `BertModel.from_pretrained`.

Thanks Gregory that the way to go indeed!