RuntimeError - invalid multinomial distribution (with replacement=False, not enough non-negative category to sample) 
Author: zeke-john
Pull request: no
Labels: name: bug

### System Info

whenever i set `do_sample=True`, i get the error: `RuntimeError('invalid multinomial distribution (with replacement=False, not enough non-negative category to sample)')`, i don't know why it is happening, but i want to use `do_sample=True` because it gives out more relevant results. Im using the Bart-Large-CNN for text summarization, with huggingface transformers version 4.16.2, any help would be greatly appreciated.

### Who can help?

@LysandreJik 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction
`
    import torch
    from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, AutoConfig

    tokenizer = AutoTokenizer.from_pretrained(model_path)
    model = AutoModelForSeq2SeqLM.from_pretrained(model_path)

    max_chunk = 1024
    current_chunk = 0 
    chunks = []
    fulltext = ''' long text here'''
    fulltext = fulltext.replace('.', '.<eos>')
    fulltext = fulltext.replace('?', '?<eos>')
    fulltext = fulltext.replace('!', '!<eos>')

    sentences = fulltext.split('<eos>')

    for sentence in sentences:  
        if len(chunks) == current_chunk + 1: 
            if len(chunks[current_chunk]) + len(sentence.split(' ')) <= max_chunk:
                chunks[current_chunk].extend(sentence.split(' '))
            else:
                current_chunk += 1
                chunks.append(sentence.split(' '))
        else:
            chunks.append(sentence.split(' '))

    for chunk_id in range(len(chunks)):
        chunks[chunk_id] = ' '.join(chunks[chunk_id])
    chink_list = []
    for chinks in chunks:
        inputs = tokenizer(str(chinks), return_tensors="pt", truncation=True)

        outputs = model.generate(inputs["input_ids"], do_sample=True)

        chunk_summary = tokenizer.decode(outputs[0])
        chunk_summary = str(chunk_summary)
        chunk_summary = chunk_summary[:-4]
        chunk_summary = chunk_summary[7:]

        chink_list.append(chunk_summary)
        summary = ' '.join(chink_list)
    print( summary)
`

### Expected behavior

no errors and for it to run normally.

Comments:
is also get this warning: `UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  next_indices = next_tokens // vocab_size`

@zeke-john, thanks for your issue! Please use tags responsibly, tagging everyone involved with GitHub won't guarantee you an answer.

We try to keep the github issues for bugs/feature requests. Could you ask your question on the [forum](https://discuss.huggingface.co) instead? We'll be unable to help you without you providing a complete (ideally small) reproducer, so without the long text in question it will be tough to find the issue for you. 

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.