TF: GPT-2 generation supports left-padding
Author: gante
Pull request: yes

# What does this PR do?

This PR does two things:
1. Enables left-padding with GPT-2 generation. 
    - It was working before only with XLA and was left as a TODO;
    - 🚨 Naturally, tests had to be changed. In the batched tests, the shortest sequence is now different, as a consequence of the correct processing of the left-padding;
    - Because we now have non-XLA left-padding, the XLA/non-XLA equivalence tests for GPT-2 now have two entries with different lengths;
    - An additional test was added, to ensure the output is the same regardless of left-padding.
2. Fix minor issues and TODOs in TF generate. In particular, I'd highlight the following:
    - All generated arrays are initialized with the `pad_token_id`, as opposed to with `0`. This was already present in `beam_search`, as one test caught it there;
    - Corrects the number of iterations in greedy search and sample -- it was one iteration short and resulting in outputs with `max_length-1` when length was the constraint (also an argument in favor by contradiction: if this change resulted in too many iterations, we would be attempting to write out of bounds of the `TensorArray`, which isn't the case)
    
___________________
Locally run slow tests: GPT-2, T5, BART, RAG, Encoder_Decoder, Vision_Encoder_Decoder, Speech2Text

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@patrickvonplaten made the changes we talked about 👍 ~~There are a couple of tests failing, but I don't think they are related to these changes (like torch tests)~~  

Cool ! Looks good to me, if possible it'd be great if @sgugger could take a quick look here since the discussed logic of how to handle "automatic creation" of the `attention_mask` is a bit universal in Transformers.

@sgugger, in short we have the following situation for the automatic attention mask creation. 

We **never** do this in the forward pass, but since a long time we have such a feature implementation implemented in PyTorch's generate: https://github.com/huggingface/transformers/blob/d156898f3b9b2c990e5963f5030a7143d57921a2/src/transformers/generation_utils.py#L490

So we won't be able to change that back in PyTorch (except for a major version). For now we do the following in PyTorch which handles the attention_mask creation correctly in 99% of the cases:
- If user doesn't provide the attention_mask **and** the padding token is in the input_ids **and** the padding token is not equal to the EOS token, we create an attention_mask automatically

This doesn't cover the edge-case where the user forwards both padding tokens and eos tokens + they are the same. Think the edge-case is really an edge case, but overall we should nudge the user to **always** provide an attention_mask if doing generate in batches. 

=> As a conclusion, we've now copied the PT logic 1-to-1 to TF generate & added a warning. After this PR is merged we should also add this warning to PT IMO. 

Does this sound good to you? 

Related: https://github.com/huggingface/transformers/pull/17444

Sounds good to me @patrickvonplaten !

Feel free to merge whenever @gante !

@gante King of TF `generate`!