model.generate function is not compatible with custom position_ids
Author: deeplxx
Pull request: no
Labels: name: bug

### System Info

The model.generate function has a flaw. When position_ids are passed in, the _update_model_kwargs_for_generation function does not handle position_ids correctly, resulting in the position not being incremented during the subsequent token sequence generation.


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

`    def _update_model_kwargs_for_generation(
        self,
        outputs,
        model_kwargs,
        is_encoder_decoder: bool = False,
        num_new_tokens: int = 1,
    ):
        # update past_key_values keeping its naming used in model code
        cache_name, cache = self._extract_past_from_model_output(outputs)
        model_kwargs[cache_name] = cache
        if getattr(outputs, "state", None) is not None:
            model_kwargs["state"] = outputs.state

        # update token_type_ids with last value
        if "token_type_ids" in model_kwargs:
            token_type_ids = model_kwargs["token_type_ids"]
            model_kwargs["token_type_ids"] = torch.cat([token_type_ids, token_type_ids[:, -1].unsqueeze(-1)], dim=-1)

        if not is_encoder_decoder:
            # update attention mask
            if "attention_mask" in model_kwargs:
                attention_mask = model_kwargs["attention_mask"]
                model_kwargs["attention_mask"] = torch.cat(
                    [attention_mask, attention_mask.new_ones((attention_mask.shape[0], 1))], dim=-1
                )
        else:
            # update decoder attention mask
            if "decoder_attention_mask" in model_kwargs:
                decoder_attention_mask = model_kwargs["decoder_attention_mask"]
                model_kwargs["decoder_attention_mask"] = torch.cat(
                    [decoder_attention_mask, decoder_attention_mask.new_ones((decoder_attention_mask.shape[0], 1))],
                    dim=-1,
                )

        if model_kwargs.get("use_cache", True):
            model_kwargs["cache_position"] = model_kwargs["cache_position"][-1:] + num_new_tokens
        else:
            past_positions = model_kwargs.pop("cache_position")
            new_positions = torch.arange(
                past_positions[-1] + 1, past_positions[-1] + num_new_tokens + 1, dtype=past_positions.dtype
            ).to(past_positions.device)
            model_kwargs["cache_position"] = torch.cat((past_positions, new_positions))

        # position_ids fix
        if model_kwargs.get("position_ids", None) is not None:
            position_ids = model_kwargs["position_ids"]
            new_position_id = position_ids[:,-1] + 1
            position_ids = torch.cat([position_ids, new_position_id.unsqueeze(1)], dim=1)
            model_kwargs["position_ids"] = position_ids

        return model_kwargs`

### Expected behavior

Properly handle custom position_ids

Comments:
cc @gante 

@deeplxx It looks like your _update_model_kwargs_for_generation function might have an issue when handling position_ids, particularly when position_ids is passed but does not correctly increment position indices.
Possible Fixes:
1. Fix position_ids Increment Logic
Ensure that the last token’s position ID increments correctly:
```python
if model_kwargs.get("position_ids", None) is not None:
    position_ids = model_kwargs["position_ids"]
    # Ensure the last token's position ID increments
    new_position_id = position_ids[:, -1] + 1
    new_position_id = new_position_id.unsqueeze(1)  # Expand dimension
    model_kwargs["position_ids"] = torch.cat([position_ids, new_position_id], dim=1)
```
2.Check cache_position Updates
Ensure that cache_position updates correctly when caching is enabled:
```python
if model_kwargs.get("use_cache", True):
    model_kwargs["cache_position"] = model_kwargs["cache_position"][-1:] + num_new_tokens
else:
    past_positions = model_kwargs.pop("cache_position")
    new_positions = torch.arange(
        past_positions[-1] + 1, past_positions[-1] + num_new_tokens + 1, dtype=past_positions.dtype
    ).to(past_positions.device)
    model_kwargs["cache_position"] = torch.cat((past_positions, new_positions))
```
Other Debugging Methods:
1. Inspect position_ids Changes
Print position_ids during generate to verify correct increments:
```python
print("Updated position_ids:", model_kwargs["position_ids"])
```
2. Try Running Without position_ids
If position_ids is not strictly required in generate, test removing it:
```python
model_kwargs.pop("position_ids", None)
```

@deeplxx Thank you for opening this issue. Can you provide a stand-alone script to reproduce it? 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.