Trouble running 'bert-base-multilingual-cased'
Author: jungwhank
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

Hi, 
I got trouble when I apply Quickstart BERT example to korean news text.

but when I run this code using model 'bert-base-multilingual-cased',

```{.python}
# Predict hidden states features for each layer
with torch.no_grad():
    # See the models docstrings for the detail of the inputs
    outputs = model(tokens_tensor, token_type_ids=segments_tensors)
    # Transformers models always output tuples.
    # See the models docstrings for the detail of all the outputs
    # In our case, the first element is the hidden state of the last layer of the Bert model
    encoded_layers = outputs[0]
# We have encoded our input sequence in a FloatTensor of shape (batch size, sequence length, model hidden dimension)
assert tuple(encoded_layers.shape) == (1, len(indexed_tokens), model.config.hidden_size)
```

 I sometimes got error like below,

```{.python}
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-39-b8e77f8ffc14> in <module>
      2 with torch.no_grad():
      3     # See the models docstrings for the detail of the inputs
----> 4     outputs = model(tokens_tensor, token_type_ids=segments_tensors)
      5     # Transformers models always output tuples.
      6     # See the models docstrings for the detail of all the outputs

//anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--> 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

//anaconda3/lib/python3.7/site-packages/transformers/modeling_bert.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask)
    712             head_mask = [None] * self.config.num_hidden_layers
    713 
--> 714         embedding_output = self.embeddings(input_ids=input_ids, position_ids=position_ids, token_type_ids=token_type_ids, inputs_embeds=inputs_embeds)
    715         encoder_outputs = self.encoder(embedding_output,
    716                                        attention_mask=extended_attention_mask,

//anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--> 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

//anaconda3/lib/python3.7/site-packages/transformers/modeling_bert.py in forward(self, input_ids, token_type_ids, position_ids, inputs_embeds)
    176             inputs_embeds = self.word_embeddings(input_ids)
    177         position_embeddings = self.position_embeddings(position_ids)
--> 178         token_type_embeddings = self.token_type_embeddings(token_type_ids)
    179 
    180         embeddings = inputs_embeds + position_embeddings + token_type_embeddings

//anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--> 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

//anaconda3/lib/python3.7/site-packages/torch/nn/modules/sparse.py in forward(self, input)
    112         return F.embedding(
    113             input, self.weight, self.padding_idx, self.max_norm,
--> 114             self.norm_type, self.scale_grad_by_freq, self.sparse)
    115 
    116     def extra_repr(self):

//anaconda3/lib/python3.7/site-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1482         # remove once script supports set_grad_enabled
   1483         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 1484     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1485 
   1486 

RuntimeError: index out of range: Tried to access index 2 out of table with 1 rows. at ../aten/src/TH/generic/THTensorEvenMoreMath.cpp:418
```

Any help will be greatly appreciated.
Thanks!


Info:
OS: MacOsX 10.14.6 (Mojave)
python : 3.7
PyTorch : 1.3.1





Comments:
I think the error was because text is greater than 512 tokens.
I got no error when text is smaller than 512 tokens.

Hi! Indeed the models have a maximum input size, which is 512 for BERT. You should have received a warning when tokenizing your sequence, but unfortunately, there isn't much more we can do to clarify this error further.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.