Fix Bug: Gemma2 the `past_key_value.update()` function has added a new parameter "sliding_window" to support the `_sliding_update` function.
Author: kkk935208447
Pull request: yes

# What does this PR do?
System Info
transformers 4.42.3

Now gemma2 model generates long text that exceeds the window size, it will report a CUDA error, which seems to be a problem with the failure of the _sliding_update function in HybridCache. The error is as follows:

```
Traceback (most recent call last):
  File "/workspace/LongQLoRA_gemma2/test_example/test.py", line 31, in <module>
    outputs = model.generate(**input_ids, do_sample=False, max_new_tokens=3000)
  File "/opt/conda/envs/llm/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/opt/conda/envs/llm/lib/python3.10/site-packages/transformers/generation/utils.py", line 1945, in generate
    result = self._sample(
  File "/opt/conda/envs/llm/lib/python3.10/site-packages/transformers/generation/utils.py", line 2693, in _sample
    outputs = self(
  File "/opt/conda/envs/llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/opt/conda/envs/llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/workspace/LongQLoRA_gemma2/PI/modeling_gemma2.py", line 1225, in forward
    outputs = self.model(
  File "/opt/conda/envs/llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/opt/conda/envs/llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/workspace/LongQLoRA_gemma2/PI/modeling_gemma2.py", line 1065, in forward
    layer_outputs = decoder_layer(
  File "/opt/conda/envs/llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/opt/conda/envs/llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/workspace/LongQLoRA_gemma2/PI/modeling_gemma2.py", line 807, in forward
    hidden_states, self_attn_weights, present_key_value = self.self_attn(
  File "/opt/conda/envs/llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/opt/conda/envs/llm/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/workspace/LongQLoRA_gemma2/PI/modeling_gemma2.py", line 709, in forward
    query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin)
  File "/workspace/LongQLoRA_gemma2/PI/modeling_gemma2.py", line 266, in apply_rotary_pos_emb
    k_embed = (k * cos) + (rotate_half(k) * sin)
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
```

After testing, the `past_key_value.update()` function has added a new parameter "sliding_window" to support the normal execution of the `_sliding_update` function.
```
key_states, value_states = past_key_value.update(key_states, value_states, self.layer_idx, cache_kwargs, 
                                                             # TODO Added the parameter `sliding_window` to support the `_sliding_update` function.
                                                             sliding_window = self.sliding_window)



@ArthurZucker
@zucchini-nlp
@gante (all others)


Comments: