Encounter an "index out of range problem"
Author: Yuejiang-li
Pull request: no

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
Hi!
I'm currently using BERT to obtain sentence embeddings for Chinese text inputs.
Things are fine for most cases. However when I am dealing with this text:
```python
weibo_content = "貌似还是没有完全修复。  http://As.international.anger.grows.over.reports.of.mass.carnage.at.the.hands.of.the.Syrian.regime.a.U.N.Security.Council.draft.resolution.condemning.Syria.failed.to.be.adopted.Saturday.after.vetowielding.members.Russia.and.China.voted.against.it.Ambassadors.from.the.other.permanent.members.of.the.council..the.United.States.France.and.the.United.Kingdom..said.they.were.furious.at.Russia.and.China.for.failing.to.halt.the.worsening.bloody.violence.that.has.consumed.the.Middle.Eastern.nation.Thirteen.Security.Council.members.voted.in.favor.of.the.resolution.The.vote.was.a.major.diplomatic.setback.for.countries.hoping.to.send.a.unified.message.to.embattled.Syrian.President.Bashar.alAssad.and.also.for.opposition.groups.that.look.toward.the.United.Nations.for.support.Those.that.have.blocked.potentially.the.last.effort.to.resolve.this.peacefully.will.have.any.future.blood.spill.on.their.hands.U.S.Ambassador.Susan.Rice.told.CNN.The.people.of.Syria.have.yet.again.been.abandoned.by.this.Council.and.by.the.international.community.Some.Syrians.have.cried.out.for.international.action.to.stop.attacks.on.civilians.more.so.after.opposition.groups.said.at.least.321.civilians.were.killed.and.hundreds.wounded.in.the.city.of.Homs.in.the.past.two.days.The.opposition.Syrian.National.Council.blamed.government.forces.for.the.attack.in.Homs.calling.it.one.of.the.most.horrific.massacres.since.the.start.of.the.Syrian.uprising.Residential.buildings.and.homes.were.randomly.and.heavily.bombed.the.group.said.The.Local.Coordination.Committees.LCC.a.Syrian.opposition.group.said.90.people.had.been.killed.in.Syria.on.Saturday.including.61.in.Homs.10.in.Idlib.and.19.in.a.Damascus.suburb.In.a.bid.to.pressure.the.government.the.group.called.for.a.twoday.civil.strike.to.start.on.Sunday.Another.opposition.group.the.Syrian.Observatory.for.Human.Rights.reported.that.48.people.were.killed.across.Syria.on.Saturday.including.six.army.defectors.and.18.members.of.the.Syrian.security.forces"
```
something goes wrong...

Okay, This text piece contains an url with many English words in it. But, I think using "bert-base-chinese" can still handle this situation.

So the following code goes like:
```python
import torch
from transformers import BertTokenizer, BertModel

tokenizer = BertTokenizer.from_pretrained('bert-base-chinese')
model = BertModel.from_pretrained('./bert-base-chinese')
model.eval()

weibo_content = "[cls]" + weibo_content + "[sep]"  # weibo_content is the target text to be extracted embeddings from
tokenized_weibo_content = tokenizer.tokenize(weibo_content) 
indexed_tokens = tokenizer.convert_tokens_to_ids(tokenized_weibo_content) 
segments_ids = [1] * len(tokenized_weibo_content) 
tokens_tensor = torch.tensor([indexed_tokens])
segments_tensors = torch.tensor([segments_ids])

with torch.no_grad(): 
    encoded_layers, _ = model(tokens_tensor, segments_tensors)
```
In the last step of the above code, I meet with the following problem:
```python
Traceback (most recent call last):
  File "C:\Users\xzzz\Anaconda3\envs\cfdstorch\lib\site-packages\IPython\core\interactiveshell.py", line 3326, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-12-c7fe4edd73d7>", line 7, in <module>
    encoded_layers, _ = model(tokens_tensor, segments_tensors)
  File "C:\Users\xzzz\Anaconda3\envs\cfdstorch\lib\site-packages\torch\nn\modules\module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\xzzz\Anaconda3\envs\cfdstorch\lib\site-packages\transformers\modeling_bert.py", line 735, in forward
    embedding_output = self.embeddings(input_ids=input_ids, position_ids=position_ids, token_type_ids=token_type_ids, inputs_embeds=inputs_embeds)
  File "C:\Users\xzzz\Anaconda3\envs\cfdstorch\lib\site-packages\torch\nn\modules\module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\xzzz\Anaconda3\envs\cfdstorch\lib\site-packages\transformers\modeling_bert.py", line 187, in forward
    position_embeddings = self.position_embeddings(position_ids)
  File "C:\Users\xzzz\Anaconda3\envs\cfdstorch\lib\site-packages\torch\nn\modules\module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\xzzz\Anaconda3\envs\cfdstorch\lib\site-packages\torch\nn\modules\sparse.py", line 114, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "C:\Users\xzzz\Anaconda3\envs\cfdstorch\lib\site-packages\torch\nn\functional.py", line 1484, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: index out of range: Tried to access index 512 out of table with 511 rows. at C:\w\1\s\tmp_conda_3.6_111945\conda\conda-bld\pytorch_1572952852006\work\aten\src\TH/generic/THTensorEvenMoreMath.cpp:418
```
Does anyone know why this happens?

Comments:
BTW, I manually downloaded the pretrained model, and save it in dir "./bert-base-chinese"

@Yuejiang-li 
To me, the error msg indicates that the seq generated after tokenization of "weibo_content"  has more than 512 tokens. The 512 is the max num of tokens in one seq allowed for the BERT model (embedding layer input size). You have to separate the "weibo_content" into at least two shorter sentences and feed them separately.

You can print "tokenized_weibo_content" to confirm.

@FacingBugs 
That's true. I forgot that point... Thank you so much!

> That's true. I forgot that point... Thank you so much!

Hi Yuejiang, Can you please share the way you have separated the content? I'm facing the same problem.

Kind regards