TypeError: LlavaLlamaForCausalLM.forward() got an unexpected keyword argument 'cache_position'
Author: Naozumi520
Pull request: no

### System Info

- `transformers` version: 4.39.0.dev0
- Platform: Windows-10-10.0.22621-SP0
- Python version: 3.11.8
- Huggingface_hub version: 0.21.3
- Safetensors version: 0.4.2
- Accelerate version: 0.27.2
- Accelerate config:    not found
- PyTorch version (GPU?): 2.2.1+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker @amyeroberts 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

1. Run llava using llava.serve.cli and libraries in my versions.
2. Ask model some questions.
3. Return error:

```

user
: hi

assistant
: Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Naozu\Desktop\AI\2.Models\Nazuna20240304\LLaVA\llava\serve\cli.py", line 128, in <module>
    main(args)
  File "C:\Users\Naozu\Desktop\AI\2.Models\Nazuna20240304\LLaVA\llava\serve\cli.py", line 98, in main
    output_ids = model.generate(
                 ^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\Desktop\AI\2.Models\Nazuna20240304\LLaVA\llava\model\language_model\llava_llama.py", line 137, in generate
    return super().generate(
           ^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\generation\utils.py", line 1597, in generate
    result = self.sample(
             ^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\transformers\generation\utils.py", line 2711, in sample
    outputs = self(
              ^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Naozu\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: LlavaLlamaForCausalLM.forward() got an unexpected keyword argument 'cache_position'
```

### Expected behavior

Everything should work just fine.

Comments:
Hey! I think you should open this issue on the `LLaVA`  repo maybe? or just update your version of `transformers`! 

Hm that's weird, cuz I install `transformers` by forking the repo, it should be the latest one...

Could you provide a `transformers` reproducer? Here it is wrapped around the `\LLaVA\llava\model\language_model\llava_llama.py` and I am not sure what is in there! 

a workaround here, add `cache_position=None` to the `forward()` method in `Class LlavaLlamaForCausalLM`

I get the same error when I Fine-Tune my llama type models since I updated transformer library

Again I need a reproducer to be able to push a fix. Also now the `cache_positions` should no be required by the model and is intialized if not passed, see: https://github.com/huggingface/transformers/blob/76a33a10923ccc1074917f6b6a1e719e626b7dc9/src/transformers/models/llama/modeling_llama.py#L980-L985.


I had the same issue when I tried with transformers version 4.38.x and 4.39.x. I installed transformers==4.37.2 and it worked.

Thanks @amankumarhal , but again, without reproducers I am having a really hard time fixing this

This repros for me with a different model with the following code. I imagine it repros with many models.

```
from transformers import TextGenerationPipeline
from transformers.models.auto import AutoModelForCausalLM, AutoTokenizer

model_path = "allenai/OLMo-1B"
model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
pipeline = TextGenerationPipeline(model=model, tokenizer=tokenizer)
output = pipeline("question: who wrote romeo and juliet? answer: ", max_new_tokens=30)
```

I believe the issue is caused by https://github.com/huggingface/transformers/commit/23db187d9223cfbd535a3a76fb518ca2c1429633 at `src/transformers/generation/utils.py` line 1940. This introduced `"cache_position"` into `model_kwargs` (as I saw when I went over the above repro with my debugger).

Oh this is using a `custom` code (remote code). The issue should be opened and fixed on the `OlMO` repo. Otherwise just wait for #29885 to be merged, which will have native support!

+1 encountered this error when trying to use olmo

+1 encountered this error.

Olmo is not part of `transformers` will be with #29885 

Workaround that got rid of the error for me was an older version of transformers
%pip install transformers==4.38


Olmo is on transformers now

Got the same error for OLMo-1B with Transformers 4.39.3

> OLMoForCausalLM.forward() got an unexpected keyword argument 'cache_position'

edit: I was using conda, Transformers 4.40.0 is not yet available. Reinstalling most recent Transformers 4.40.0 via `pip` led to the following error mesage when importing `hf_olmo`: 

`ValueError: 'olmo' is already used by a Transformers config, pick another name.`

Please use `-hf` versions of OLMo (e.g. https://huggingface.co/allenai/OLMo-1.7-7B-hf) for Transformer 4.40.0 onwards (OLMo just got integrated into transformers!).

I also added an issue in the LLAva repo https://github.com/haotian-liu/LLaVA/issues/1448

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Transformers 4.37.2 OK！

> Transformers 4.37.2 OK！

it works!

Closing then! Thanks 🤗 

> a workaround here, add `cache_position=None` to the `forward()` method in `Class LlavaLlamaForCausalLM`

It works for me. Thanks!!!

> a workaround here, add `cache_position=None` to the `forward()` method in `Class LlavaLlamaForCausalLM`

It works for me!

> a workaround here, add `cache_position=None` to the `forward()` method in `Class LlavaLlamaForCausalLM`

helpful~

> a workaround here, add `cache_position=None` to the `forward()` method in `Class LlavaLlamaForCausalLM`

Thanks! I fixed my problem by doing so!