Additional option for text generation when setting num_beam_groups
Author: hukuda222
Pull request: no

### Feature request

I propose to add `num_return_sequences_per_groups` as an argument to `generate` function of `transformers.GenerationMixin`. Setting this will output `num_return_sequences_per_groups` sentences each group. Use cases are as follows:

code:
```python
outputs = model.generate(
    tokenizer.encode(text, return_tensors="pt", max_length=512),
    num_beam_groups=3,
    num_beams=12,
    diversity_penalty=1.0,
    num_return_sequences_per_groups=2,
)
for output in outputs:
    print(tokenizer.decode(output, skip_special_tokens=True))
```

output:
```
A flock of birds flying over the ocean.
A flock of birds flying over a beach.
Birds flying over the water in the sun.
Birds flying the water near a mountain.
Several birds are flying over a body of water.
Several birds flying over a body of water.
```

The example referred to https://arxiv.org/abs/1610.02424 .

### Motivation

As shown below, the output may have little difference when `num_beam_groups` and `num_beams` have the same values. 

code:
```python
from transformers import (
    AutoTokenizer,
    AutoModelForSeq2SeqLM,
)

model = AutoModelForSeq2SeqLM.from_pretrained("facebook/bart-large-xsum")
tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-xsum")
text = "The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration."
outputs = model.generate(
    tokenizer.encode(text, return_tensors="pt", max_length=512),
    num_beam_groups=2,
    num_beams=2,
    diversity_penalty=1000000.0,
    num_return_sequences=2,
)
for output in outputs:
    print(tokenizer.decode(output, skip_special_tokens=True))
```
output:
```
A number Of research projects have investigated the role of the brain's encoder and decoder in the control of the encoded sequences.
A number Of research projects have investigated the role of the brain's encoder and decoder in the control of the encoded sequences..
```

This problem occurs because num_beam*2 is searched as an implementation of beam search. Such output is undesirable.

This example is only for clarity. Even in the general case, the current implementation does not guarantee diversity because the output is ordered by score. Therefore, I would like to enable a variety of outputs with this option.

### Your contribution

If it looks good, I will implement it.

Comments:
cc @gante 

Hey @hukuda222 

`generate` is already a configuration behemoth, and we would be adding one more flag. By default, we are reluctant to add more flags unless the benefits are large OR there is demand for the option. As such, I'm going to propose the same as I do in similar issues ([e.g.](https://github.com/huggingface/transformers/issues/22168#issuecomment-1477998997))!

If this comment gets 10 reactions/this issue gets mentioned 10 times, then it means that folks have been searching for this feature. In that case, I'll greenlight the suggestion, and let's add it to the codebase. That way, we can balance HF's limited maintenance resources with actual feature demand! (Whoever does the 10th react, plz tag me)

@hukuda222 does that sound good to you?

@gante Sounds good. Thank you for your quick response.

@gante 
Sorry for the delay. I thought the output I presented earlier might be a bug in the current code. Diverse beam search is a method that generates  `num_beams//num_beam_groups` sentences for each group independently. However, the current code uses one BeamHypotheses shared by all groups. Therefore, group A will generate two sentences before group B outputs a sentence.

https://github.com/huggingface/transformers/blob/ad78d9597b224443e9fe65a94acc8c0bc48cd039/src/transformers/generation/beam_search.py#L178-L186

This is a problem that can be solved by creating as many BeamHypotheses as there are groups. I would like to address this inconvenience in the form of a modification to the diverse beam search implementation, rather than adding an option. If you don't mind, could you give me your opinion?

Hey @hukuda222 👋 

I've had a deeper look at group beam search, and it doesn't not seem to be working properly. For instance, the snippet below produces the same sequence on all beams, and that should not happen (each beam should generate different continuations).

I don't have the bandwidth to fix it immediately, so if you're able to contribute we'd deeply appreciate 🙌 Since it is broken (and thus no retrocompatibility needs to be respected), feel free to also change the behavior of `num_return_sequences` in group beam search to prioritize returning from different beams, which makes more sense.

___________________________________
```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")
inputs = tokenizer(["The full name of Donald is Donald"], return_tensors="pt")

outputs = model.generate(**inputs, num_beams=4, num_beam_groups=4, num_return_sequences=4)
print("\n".join(tokenizer.batch_decode(outputs, skip_special_tokens=True)))
# Outputs the following sequence 4 times. Each beam should return different sequences.
# The full name of Donald is Donald J. Trump Jr. The full name of Donald is Donald J
```

@gante 
Thanks for doing the research. I will send PR as soon as I can fix it.