Repository: huggingface/evaluate
Issue: IndexError: index out of range in self when computing perplexity with German GPT-2
Root cause: Validation Gap

I followed [your tutorial](https://huggingface.co/spaces/evaluate-metric/perplexity) on computing perplexity of list of sentences. I used the German GPT-2 model `dbmdz/german-gpt2`.

My code:
```
from evaluate import load

perplexity = load("perplexity", module_type="metric")
input_texts = ["lorem ipsum", "Happy birthday!", "Happy Birthday!", "Bienvenue", "My name is John."]
perplexity_results = perplexity.compute(predictions=input_texts, model_id="dbmdz/german-gpt2", add_start_token=False, device="cpu")

```

Error:
```
Traceback (most recent call last):
  File "/home/izaskr/text_analysis/_testing.py", line 34, in <module>
    perplexity_results = perplexity.compute(predictions=input_texts, model_id="dbmdz/german-gpt2", add_start_token=False, device="cpu")
  File "/home/izaskr/anaconda3/envs/analysis_daf/lib/python3.10/site-packages/evaluate/module.py", line 444, in compute
    output = self._compute(**inputs, **compute_kwargs)
  File "/home/izaskr/.cache/huggingface/modules/evaluate_modules/metrics/evaluate-metric--perplexity/23c7f61948a6e59ecbd1e92ea827a525b0a412dcef1f1a5bec3dc007452c8fa8/perplexity.py", line 177, in _compute
    out_logits = model(encoded_batch, attention_mask=attn_mask).logits
  File "/home/izaskr/anaconda3/envs/analysis_daf/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/izaskr/anaconda3/envs/analysis_daf/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1063, in forward
    transformer_outputs = self.transformer(
  File "/home/izaskr/anaconda3/envs/analysis_daf/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/izaskr/anaconda3/envs/analysis_daf/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 849, in forward
    inputs_embeds = self.wte(input_ids)
  File "/home/izaskr/anaconda3/envs/analysis_daf/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/izaskr/anaconda3/envs/analysis_daf/lib/python3.10/site-packages/torch/nn/modules/sparse.py", line 158, in forward
    return F.embedding(
  File "/home/izaskr/anaconda3/envs/analysis_daf/lib/python3.10/site-packages/torch/nn/functional.py", line 2199, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self
```

When I use the English GPT-2 model `gpt2`, there's no error.

This error has nothing to do with `evaluate`. It is a bug inside the german model itself.
`IndexError: index out of range in self`: this error appear when your encoded token ids are larger than your embeddings size.
For instance, in this case:
```
'input_ids': tensor([[10096,   305,   300,  3204,   335],
        [44972,   296, 32060,  4579,     5],
        [44972,   304, 32060,  4579,     5],
        [   38,   427,  1547,   698, 50265],
        [13982, 33898,  1568,  3695,    18]])
```
and the embedding size (you can see this through `model.transformer`): 50265.
indices cannot be higher than embedding_dim-1, which is 50264, since the embedding layer is working as a lookup table. The input is used to index the corresponding embedding vector.