model.generate occurs error:  generation_beam_search
Author: Albert-Ma
Pull request: no

## Environment info
- `transformers` version: 4.2.1
- Platform: Linux-3.10.0-1127.18.2.el7.x86_64-x86_64-with-glibc2.10
- Python version: 3.8.8
- PyTorch version (GPU?): 1.5.0 (True)
- Tensorflow version (GPU?): 2.4.1 (False)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help
@patrickvonplaten, @patil-suraj


## Information

Model I am using (Bert, XLNet ...): bart-base

The problem arises when using:
```
/opt/conda/conda-bld/pytorch_1587428207430/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:87: binarySearchForMultinomial: block: [39,0,0], thread: [0,3,0] Ass
ertion `cumdist[size - 1] > static_cast<scalar_t>(0)` failed.
/opt/conda/conda-bld/pytorch_1587428207430/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:87: binarySearchForMultinomial: block: [23,0,0], thread: [0,1,0] Ass
ertion `cumdist[size - 1] > static_cast<scalar_t>(0)` failed.
/opt/conda/conda-bld/pytorch_1587428207430/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:87: binarySearchForMultinomial: block: [29,0,0], thread: [0,2,0] Ass
ertion `cumdist[size - 1] > static_cast<scalar_t>(0)` failed.
/opt/conda/conda-bld/pytorch_1587428207430/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:87: binarySearchForMultinomial: block: [39,0,0], thread: [0,0,0] Ass
ertion `cumdist[size - 1] > static_cast<scalar_t>(0)` failed.
/opt/conda/conda-bld/pytorch_1587428207430/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:87: binarySearchForMultinomial: block: [23,0,0], thread: [0,2,0] Ass
ertion `cumdist[size - 1] > static_cast<scalar_t>(0)` failed.
/opt/conda/conda-bld/pytorch_1587428207430/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:87: binarySearchForMultinomial: block: [39,0,0], thread: [0,1,0] Ass
ertion `cumdist[size - 1] > static_cast<scalar_t>(0)` failed.
/opt/conda/conda-bld/pytorch_1587428207430/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:87: binarySearchForMultinomial: block: [23,0,0], thread: [0,3,0] Ass
ertion `cumdist[size - 1] > static_cast<scalar_t>(0)` failed.
/opt/conda/conda-bld/pytorch_1587428207430/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:87: binarySearchForMultinomial: block: [39,0,0], thread: [0,2,0] Ass
ertion `cumdist[size - 1] > static_cast<scalar_t>(0)` failed.
/opt/conda/conda-bld/pytorch_1587428207430/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:87: binarySearchForMultinomial: block: [23,0,0], thread: [0,0,0] Ass
ertion `cumdist[size - 1] > static_cast<scalar_t>(0)` failed.
/opt/conda/conda-bld/pytorch_1587428207430/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:87: binarySearchForMultinomial: block: [28,0,0], thread: [0,1,0] Ass
ertion `cumdist[size - 1] > static_cast<scalar_t>(0)` failed.
/opt/conda/conda-bld/pytorch_1587428207430/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:87: binarySearchForMultinomial: block: [22,0,0], thread: [0,1,0] Ass
ertion `cumdist[size - 1] > static_cast<scalar_t>(0)` failed.
/opt/conda/conda-bld/pytorch_1587428207430/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:87: binarySearchForMultinomial: block: [56,0,0], thread: [0,0,0] Ass
ertion `cumdist[size - 1] > static_cast<scalar_t>(0)` failed.
/opt/conda/conda-bld/pytorch_1587428207430/work/aten/src/ATen/native/cuda/MultinomialKernel.cu:87: binarySearchForMultinomial: block: [44,0,0], thread: [0,3,0] Ass
ertion `cumdist[size - 1] > static_cast<scalar_t>(0)` failed.
....... similar as above

Traceback (most recent call last):                                                                                                                        [56/1924]
  File "run_eval.py", line 172, in <module>
    run_generate(verbose=True)
  File "run_eval.py", line 133, in run_generate
    runtime_metrics = generate_summaries_or_translations(
  File "run_eval.py", line 67, in generate_summaries_or_translations
    summaries = model.generate(
  File "/home/xxx/anaconda3/envs/xxx/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)
  File "/data/xxx/transformers/src/transformers/generation_utils.py", line 986, in generate
    return self.beam_sample(
  File "/data/xxx/transformers/src/transformers/generation_utils.py", line 1894, in beam_sample
    beam_outputs = beam_scorer.process(
  File "/data/xxx/transformers/src/transformers/generation_beam_search.py", line 218, in process
    if self._done[batch_idx]:
RuntimeError: CUDA error: device-side assert triggered
```

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: seq2seq

## To reproduce

Steps to reproduce the behavior:

```
python run_eval.py \
    --model_name ${MODEL_DIR} \
    --input_path $DATA_DIR/val.src \
    --save_path $DATA_DIR/xxx.txt \
    --task summarization \
    --device cuda:0 \
    --bs 50 \
    --min_length 2 \
    --max_length 32 \
    --do_sample True \
    --top_k 10 \
    --num_return_sequences 5
```

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I meet the same problem, have you solved it? @Albert-Ma 

@qshi95 - could you provide the full command to reproduce the error?

I find the reason causing this error is that the size of input ids is out of range. Sorry for disturb.