microsoft/Phi-3.5-mini-instruct not working with FA2 due to position_ids
Author: BramVanroy
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.44.0
- Platform: Linux-5.14.0-427.20.1.el9_4.x86_64-x86_64-with-glibc2.34
- Python version: 3.10.13
- Huggingface_hub version: 0.26.3
- Safetensors version: 0.4.5
- Accelerate version: 0.33.0
- Accelerate config:    - compute_environment: LOCAL_MACHINE
        - distributed_type: MULTI_GPU
        - mixed_precision: bf16
        - use_cpu: False
        - debug: False
        - num_processes: 2
        - machine_rank: 0
        - num_machines: 1
        - gpu_ids: all
        - rdzv_backend: static
        - same_network: True
        - main_training_function: main
        - enable_cpu_affinity: False
        - downcast_bf16: no
        - tpu_use_cluster: False
        - tpu_use_sudo: False
        - tpu_env: []
- PyTorch version (GPU?): 2.4.0+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA GeForce RTX 3090


### Who can help?

@ArthurZucker @sunmarc

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('microsoft/Phi-3.5-mini-instruct')
model = AutoModelForCausalLM.from_pretrained(
    'microsoft/Phi-3.5-mini-instruct',
    device_map={"": 0},
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2"  # Commenting this line will remove the issue
)
model.eval()

model = torch.compile(model)
# print(model.config.eos_token_id)
print(tokenizer.eos_token_id)

gen_kwargs = {
    "max_new_tokens": 1,
    "pad_token_id": tokenizer.eos_token_id,
    "eos_token_id": tokenizer.eos_token_id,
    "do_sample": False,
    "top_p": None,
    "top_k": None,
    "temperature": None,
}
ctx_len = 128
device = model.device
with torch.inference_mode():
    inputs = {
        "input_ids": torch.LongTensor([[42] * ctx_len]).to(device),
        "attention_mask": torch.ones(1, ctx_len).to(device),
        "position_ids": torch.arange(ctx_len).unsqueeze(0).to(device),
    }

    model.generate(**inputs, **gen_kwargs)

```

Error:

```
Traceback (most recent call last):
  File "/home/pricie/vanroy/.config/JetBrains/PyCharm2024.1/scratches/scratch_68.py", line 35, in <module>
    model.generate(**inputs, **gen_kwargs)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/transformers/generation/utils.py", line 2024, in generate
    result = self._sample(
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/transformers/generation/utils.py", line 2982, in _sample
    outputs = self(**model_inputs, return_dict=True)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/transformers/models/phi3/modeling_phi3.py", line 1247, in forward
    outputs = self.model(
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/transformers/models/phi3/modeling_phi3.py", line 1051, in forward
    layer_outputs = decoder_layer(
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/transformers/models/phi3/modeling_phi3.py", line 788, in forward
    attn_outputs, self_attn_weights, present_key_value = self.self_attn(
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/transformers/models/phi3/modeling_phi3.py", line 543, in forward
    cos, sin = self.rotary_emb(value_states, seq_len=rotary_seq_len)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/local/vanroy/clin34-benchmarks/.venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
TypeError: Phi3LongRoPEScaledRotaryEmbedding.forward() missing 1 required positional argument: 'position_ids'
```

### Expected behavior

No error

Comments:
Hey @BramVanroy , this is already fixed and working in v4.45. Can you update with `pip install -U transformers`

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.