🐛Diverse Beam Search BUG
Author: Hannibal046
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
2022-04-15 20:47:39.551372: I tensorflow/core/platform/cpu_feature_guard.cc:145] This TensorFlow binary is optimized with Intel(R) MKL-DNN to use the following CPU instructions in performance critical operations:  SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA
To enable them in non-MKL-DNN operations, rebuild TensorFlow with the appropriate compiler flags.
2022-04-15 20:47:39.605017: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2494135000 Hz
2022-04-15 20:47:39.605529: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55f0c01a5f30 executing computations on platform Host. Devices:
2022-04-15 20:47:39.605558: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.19.0.dev0
- Platform: Linux-5.4.32-1-tlinux4-0001-x86_64-with-centos-8.2.2.2004-Core
- Python version: 3.7.6
- Huggingface_hub version: 0.5.1
- PyTorch version (GPU?): 1.2.0 (True)
- Tensorflow version (GPU?): 2.0.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <no>
- Using distributed or parallel set-up in script?: <no>


### Who can help

@patrickvonplaten @narsil


## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-de")
model = AutoModelForSeq2SeqLM.from_pretrained("Helsinki-NLP/opus-mt-en-de")
input = "Machine learning is great, isn't it?"
input_ids = tokenizer.encode(input, return_tensors="pt")
outputs = model.generate(input_ids,num_return_sequences=5, num_beam_groups=5, diversity_penalty=1.0, num_beams=5)
decoded = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(decoded)
```

```
/home/chengxin/anaconda3/envs/simcls/lib/python3.7/site-packages/transformers/generation_beam_search.py:197: UserWarning: Passing `max_length` to BeamSearchScorer is deprecated and has no effect. `max_length` should be passed directly to `beam_search(...)`, `beam_sample(...)`, or `group_beam_search(...)`.
  "Passing `max_length` to BeamSearchScorer is deprecated and has no effect. "
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_2692803/1384243548.py in <module>
      5 input_ids = tokenizer.encode(input, return_tensors="pt")
      6 # gen_kwargs = dict(num_return_sequences=16, num_beam_groups=5, diversity_penalty=1.0, num_beams=16)
----> 7 outputs = model.generate(input_ids,num_return_sequences=5, num_beam_groups=5, diversity_penalty=1.0, num_beams=5)
      8 decoded = tokenizer.decode(outputs[0], skip_special_tokens=True)
      9 print(decoded)

~/anaconda3/envs/simcls/lib/python3.7/site-packages/torch/autograd/grad_mode.py in decorate_no_grad(*args, **kwargs)
     47         def decorate_no_grad(*args, **kwargs):
     48             with self:
---> 49                 return func(*args, **kwargs)
     50         return decorate_no_grad
     51 

~/anaconda3/envs/simcls/lib/python3.7/site-packages/transformers/generation_utils.py in generate(self, inputs, max_length, min_length, do_sample, early_stopping, num_beams, temperature, top_k, top_p, typical_p, repetition_penalty, bad_words_ids, force_words_ids, bos_token_id, pad_token_id, eos_token_id, length_penalty, no_repeat_ngram_size, encoder_no_repeat_ngram_size, num_return_sequences, max_time, max_new_tokens, decoder_start_token_id, use_cache, num_beam_groups, diversity_penalty, prefix_allowed_tokens_fn, logits_processor, renormalize_logits, stopping_criteria, constraints, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, forced_bos_token_id, forced_eos_token_id, remove_invalid_values, synced_gpus, exponential_decay_length_penalty, **model_kwargs)
   1432                 return_dict_in_generate=return_dict_in_generate,
   1433                 synced_gpus=synced_gpus,
-> 1434                 **model_kwargs,
   1435             )
   1436 

~/anaconda3/envs/simcls/lib/python3.7/site-packages/transformers/generation_utils.py in group_beam_search(self, input_ids, beam_scorer, logits_processor, stopping_criteria, max_length, pad_token_id, eos_token_id, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, synced_gpus, **model_kwargs)
   2877 
   2878                 next_token_scores_processed = logits_processor(
-> 2879                     group_input_ids, next_token_scores, current_tokens=current_tokens, beam_group_idx=beam_group_idx
   2880                 )
   2881                 next_token_scores = next_token_scores_processed + beam_scores[batch_group_indices].unsqueeze(-1)

~/anaconda3/envs/simcls/lib/python3.7/site-packages/transformers/generation_logits_process.py in __call__(self, input_ids, scores, **kwargs)
     88                         f"{processor.__class__} are passed to the logits processor."
     89                     )
---> 90                 scores = processor(input_ids, scores, **kwargs)
     91             else:
     92                 scores = processor(input_ids, scores)

~/anaconda3/envs/simcls/lib/python3.7/site-packages/transformers/generation_logits_process.py in __call__(self, input_ids, scores, current_tokens, beam_group_idx)
    586             ]
    587             token_frequency = torch.bincount(previous_group_tokens, minlength=vocab_size).to(scores.device)
--> 588             scores[batch_idx * group_size : (batch_idx + 1) * group_size] -= self._diversity_penalty * token_frequency
    589 
    590         return scores

RuntimeError: expected device cpu and dtype Float but got device cpu and dtype Long
```
<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->


<!-- A clear and concise description of what you would expect to happen. -->


Comments:
Hey @Hannibal046, 

I cannot reproduce the bug, but I also don't think that we are still supporting PyTorch 1.2.0. Could you upgrade to PyTorch library to a newer version and try again?

Hi @patrickvonplaten ,
Thanks your time. I upgrade my PyTorch version, and the bug is gone.