[Bug Fix] Beam search example in docs fails & a fix (integrating `max_length` in `BeamScorer.finalize()`)
Author: cwkeam
Pull request: yes

# [Bug Fix] Exact beam search example in docs fails & a fix (regarding `max_length` in `BeamScorer.finalize()`)

Fixes a bug found while developing #15416 
Specifically mentioned in number three of https://github.com/huggingface/transformers/pull/15416#issuecomment-1028550095

## Example in master documentation fails

All example code in the documentation should not throw an error when copied & pasted & run directly.
However, the following example found [here in the huggingface documentation](https://huggingface.co/docs/transformers/v4.16.2/en/main_classes/model#transformers.generation_utils.GenerationMixin.beam_search) fails due to what seems like a recent change with internal workings of `max_length`.

```python
tokenizer = AutoTokenizer.from_pretrained("t5-base")
model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")

encoder_input_str = "translate English to German: How old are you?"
encoder_input_ids = tokenizer(encoder_input_str, return_tensors="pt").input_ids

# lets run beam search using 3 beams
num_beams = 3
# define decoder start token ids
input_ids = torch.ones((num_beams, 1), device=model.device, dtype=torch.long)
input_ids = input_ids * model.config.decoder_start_token_id

# add encoder_outputs to model keyword arguments
model_kwargs = {
    "encoder_outputs": model.get_encoder()(
        encoder_input_ids.repeat_interleave(num_beams, dim=0), return_dict=True
    )
}

# instantiate beam scorer
beam_scorer = BeamSearchScorer(
    batch_size=1,
    num_beams=num_beams,
    device=model.device,
)

# instantiate logits processors
logits_processor = LogitsProcessorList(
    [
        MinLengthLogitsProcessor(5, eos_token_id=model.config.eos_token_id),
    ]
)

outputs = model.beam_search(input_ids, beam_scorer, logits_processor=logits_processor, **model_kwargs)
outputs = tokenizer.batch_decode(outputs, skip_special_tokens=True)

self.assertListEqual(outputs, ["Wie alt bist du?"])
```

Running this with:
```
pytest tests/test_generation_utils.py::GenerationIntegrationTests::test_beam_search_example_integration
```

leads to the following error:
```
        # select the best hypotheses
        sent_lengths = input_ids.new(batch_size * self.num_beam_hyps_to_keep)
        best = []
        best_scores = torch.zeros(batch_size * self.num_beam_hyps_to_keep, device=self.device, dtype=torch.float32)
    
        # retrieve best hypotheses
        for i, beam_hyp in enumerate(self._beam_hyps):
            sorted_hyps = sorted(beam_hyp.beams, key=lambda x: x[0])
            for j in range(self.num_beam_hyps_to_keep):
                best_hyp_tuple = sorted_hyps.pop()
                best_score = best_hyp_tuple[0]
                best_hyp = best_hyp_tuple[1]
                sent_lengths[self.num_beam_hyps_to_keep * i + j] = len(best_hyp)
    
                # append to lists
                best.append(best_hyp)
                best_scores[i * self.num_beam_hyps_to_keep + j] = best_score
    
        # prepare for adding eos
>       sent_max_len = min(sent_lengths.max().item() + 1, max_length)
E       TypeError: '<' not supported between instances of 'NoneType' and 'int'

src/transformers/generation_beam_search.py:333: TypeError

```

To ensure that this issue is completely resolved, part of this PR includes a test function that directly copies the example in the documentation.

## The Bug Fix

The suggested testing code `test_beam_search_example_integration(self)` is **taken directly from the current huggingface documentation** for `GenerationMixin.beam_search` and it fails, because `max_length` can be set to None unless otherwise specified, and it throws an error saying that you can't compare a `Tensor` to a `NoneType`.

[From `BeamSearchScorer.finalize()`](https://github.com/huggingface/transformers/blob/master/src/transformers/generation_beam_search.py#L333),

```python
sent_max_len = min(sent_lengths.max().item() + 1, max_length) # ERROR 1
decoded: torch.LongTensor = input_ids.new(batch_size * self.num_beam_hyps_to_keep, sent_max_len)
if sent_lengths.min().item() != sent_lengths.max().item():
    assert pad_token_id is not None, "`pad_token_id` has to be defined"
    decoded.fill_(pad_token_id)

for i, hypo in enumerate(best):
    decoded[i, : sent_lengths[i]] = hypo
    if sent_lengths[i] < max_length: # ERROR 2
        decoded[i, sent_lengths[i]] = eos_token_id
```

So I changed that to:

```python
sent_lengths_max = sent_lengths.max().item() + 1
sent_max_len = min(sent_lengths_max, max_length) if max_length is not None else sent_lengths_max
decoded: torch.LongTensor = input_ids.new(batch_size * self.num_beam_hyps_to_keep, sent_max_len)
# shorter batches are padded if needed
if sent_lengths.min().item() != sent_lengths.max().item():
    assert pad_token_id is not None, "`pad_token_id` has to be defined"
    decoded.fill_(pad_token_id)

# fill with hypotheses and eos_token_id if the latter fits in
for i, hypo in enumerate(best):
    decoded[i, : sent_lengths[i]] = hypo
    if sent_lengths[i] < sent_max_len:
        decoded[i, sent_lengths[i]] = eos_token_id
```

Which now allows the test to pass.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@patrickvonplaten 




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_15555). All of your documentation changes will be reflected on that endpoint.

@patrickvonplaten done!

Pinging @patrickvonplaten here as well for merge for approved changes