Static Cache: no mandatory `cache_positions` input
Author: gante
Pull request: yes

# What does this PR do?

Removes the hard requirement of `cache_position` from the public model classes (e.g. classes that can be loaded with `AutoModel`). Its contents can be derived from the cache's `get_seq_length()`.

________________________________________________________
### Performance and Quality checks

Slow tests run, getting the same outcome as in `main`:
- [x] `RUN_SLOW=1 py.test tests/models/llama/test_modeling_llama.py -vv`
- [x] `RUN_SLOW=1 py.test tests/models/gemma/test_modeling_gemma.py -vv`
- [x] `RUN_SLOW=1 py.test tests/test_cache_utils.py -vv` [Note: two new tests were added here]

👉 Note that these tests ensure that `torch.compile` works, with and without `attention_mask` being passed.

Local benchmark (RTX3090, tiny llama) -- no changes
(main)
![Screenshot 2024-02-22 at 11 54 26](https://github.com/huggingface/transformers/assets/12240844/ec452847-4a4d-4aec-8063-2a6017a62857)

(this pr)
![Screenshot 2024-02-22 at 17 57 13](https://github.com/huggingface/transformers/assets/12240844/199edee2-1909-4045-9414-5c9975788493)


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29221). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @fxmarty :)

No specific opinon, just that using `cache_position` in the modeling code when `use_cache=False` is not very intuitive to me: https://github.com/huggingface/transformers/blob/75ed76eceaf9b20c7ec37395e4f5d491135186f9/src/transformers/models/llama/modeling_llama.py#L975-L983

Only thing I think: `cache_position` (if it remains) should be a strictly optional argument & the model should work without for BC.

EDIT -- before you read why I believe removing `cache_positions` from the signature is an important change for our users, have a look at the current diff :)

✅ much simpler computations (@ArthurZucker addresses your concerns, although the previous version also did not introduce slowdowns)
✅ working `StaticCache.get_seq_length()`, which removes a few exceptions 
✅ no exception is raised when an input tensor is missing, with the `StaticCache` (with `cache_positions` on `main`, `position_ids` in the previous commit)
[There are still a few if/elses for cache-type-dependent logic in the model, but #29180 will standardize the cache API so that all caches match the static cache]

<details>
  <summary>Updated performance numbers (i.e. still no degradation in throughput)</summary>
  
![Screenshot 2024-02-23 at 19 18 56](https://github.com/huggingface/transformers/assets/12240844/13c59e77-70cc-4713-9f31-fd0ef390bc01)
  
</details>

______________________________________________________________________________
> No specific opinon, just that using cache_position in the modeling code when use_cache=False is not very intuitive to me:

@fxmarty I agree here, that it should be renamed :) e.g. `data_position` would be more appropriate, as it may be used without cache. Precise variable names help us in the long run.

> IMO we should move to having cache_positions in generate as this is more explicit, easier to maintain and less error prone.
More than that, these operation are input dependent while being vectorized they add complexity when there should not be!

@ArthurZucker I strongly disagree, more redundant inputs result in more bugs 🐛  From the full `input_ids` and `pad_token_id` we can derive `attention_mask`, `position_ids`, and `cache_positions` -- it's not like we are giving more options to the users, there is only one set of valid tensors given the first two. In other words, we give users three different paths to get things wrong! As a result, we have to add safety warnings and exceptions, such as:
1. [warning if there is a pad token in the input but no attention mask](https://github.com/huggingface/transformers/blob/c8d98405a8f7b0e5d07391b671dcc61bb9d7bad5/src/transformers/models/gpt2/modeling_gpt2.py#L775C18-L775C55)
2. [raising an exception if cache positions are not passed](https://github.com/huggingface/transformers/blob/c8d98405a8f7b0e5d07391b671dcc61bb9d7bad5/src/transformers/models/llama/modeling_llama.py#L977)
3. [raise an exception if the attention mask is not of the right type](https://github.com/huggingface/transformers/blob/c8d98405a8f7b0e5d07391b671dcc61bb9d7bad5/src/transformers/models/gpt2/modeling_gpt2.py#L261)
4. [warning if both attention mask and pad token are missing in generate](https://github.com/huggingface/transformers/blob/c8d98405a8f7b0e5d07391b671dcc61bb9d7bad5/src/transformers/generation/utils.py#L1360)
(plus other error-prone combinations that we are not even checking 😬)

This is arguably much more work than enforcing the correct pairing in the models themselves! On top of that, other functions like `generate` or `pipeline` have to create the correct logic anyways (e.g. [create attn mask](https://github.com/huggingface/transformers/blob/c8d98405a8f7b0e5d07391b671dcc61bb9d7bad5/src/transformers/generation/utils.py#L469); [create position ids](https://github.com/huggingface/transformers/blob/c8d98405a8f7b0e5d07391b671dcc61bb9d7bad5/src/transformers/models/gpt2/modeling_gpt2.py#L1018)), so why not push the logic to the models and prevent these bugs from occurring everywhere? 🤗 

My goal with this PR is to iron out future sources of bugs before we roll out these changes to other models. Better than well-explained interfaces is... no need for that interface at all! Without these changes, users have to learn how to prepare a new input tensor to use static caches.

In terms of complexity, it will always exist. The difference is whether it exists outside `generate` (=users can get it wrong, one more argument in `generate`), inside `generate` (=we have to maintain the logic, users can get custom generation loops wrong, one more argument in `forward`), or inside `forward` (=we have to maintain the logic). In my experience, if some input can be used incorrectly, it WILL be used incorrectly. Having few bugs is one of the strong points of `transformers`, which we should strive to keep. [In the same direction, I think we should get rid of `attention_mask` and `position_ids`, but that would be a breaking change 😉 There are so many issues in this repo due to these variables 😭  I remember that at some point you were also trying to get rid of `position_ids`]

In terms of performance, we can see that it is negligible in eager forwards and none in compiled static forwards. Again, if not implemented here, it will be implemented in `generate`, resulting in the same net performance.

@ArthurZucker as we talked on Slack, the latest set of commits makes `cache_position` follow the same logic as set for `attention_mask` or `position_ids`: enable both explicit [full control of forward] and implicit [lazy usage of forward] usage 🤗 

NOTE: ignore the changes in `prepare_inputs_for_generation` for now, from what I'm seeing while enabling `torch.compile` we will need more touches there!

In summary, this PR:
1. Enables implicit `cache_position`, compatible with `torch.compile`;
2. Fixes `StaticCache.get_seq_length()` (needed for implicit `cache_position` in `prepare_inputs_for_generation`);
3. Deprecates the `seen_tokens` attribute in the other caches, as it is redundant with `cache_position`;
4. Slicing in `prepare_inputs_for_generation` with `StaticCache` now shares the original path, as opposed to having custom logic (a bunch of TODOs were added here, only feasible after future PRs);
5. Adds docstrings for `cache_position` in `forward`;
6. Adds several tests related to `cache_position`, to ensure it is working as expected.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@gante could you rebase / fix meges and I ll review!

Closing this PR and other cache PRs, as we want to move in the opposite direction (static cache behaving like the other caches)