No `past_key_values` argument for RobertaForMaskedLM, RobertaForTokenClassification, etc.
Author: simonlevine
Pull request: no

### System Info

@ArthurZucker @younesbelkada 

RoBERTa's encoder allows for `past_key_values` to be passed, but RobertaForMaskedLM / RobertaForTokenClassification `forward()` doesn't. As a result, prefix tuning (in PEFT) fails as the forward signature `past_key_values` . I'd suggest adding encoder kwargs or similar.

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Trying to run Prefix Tuning on RoBERTa (for feature extraction, etc.) results in this issue.

### Expected behavior

Expect the example to run given that RoBERTa is supported.

Comments:
@ArthurZucker I'm not sure if this is just a miss-out (maybe it's not as `MaskedLMOutput/TokenClassifierOutput` does not have a `past-key-value` arg), but `past_key_values` are of course necessary for PEFT, a quick fix would be to pass `past_key_values` through forward. 
https://github.com/huggingface/transformers/blob/2f3ea08a077ba3133fa8a604b22436cad250b055/src/transformers/models/roberta/modeling_roberta.py#L1059-L1073
and through the model as well.
https://github.com/huggingface/transformers/blob/2f3ea08a077ba3133fa8a604b22436cad250b055/src/transformers/models/roberta/modeling_roberta.py#L1084-L1095
with changes to *modeling_output*.
https://github.com/huggingface/transformers/blob/2f3ea08a077ba3133fa8a604b22436cad250b055/src/transformers/modeling_outputs.py#L725-L728


Hey! I am a bit confused here, `past_key_values` are usually only needed when you perform **generation** using the `generate` function with either an *encoder-decoder* or a *decoder*.  Pretty sure the task you are referring to do not need this no?   

> Hey! I am a bit confused here, `past_key_values` are usually only needed when you perform **generation** using the `generate` function with either an *encoder-decoder* or a *decoder*.  Pretty sure the task you are referring to do not need this no?   

Yes, but I think one should be able to use PEFT with bidirectional models for feature extraction etc. so either the PEFT implementation, documentation (mentioning BERT/RoBERTa), or function signature is incorrect. If the past key values aren't needed then the inspection of the base model's forward shouldn't check for it.

As @ArthurZucker said, `past_key_values` is not for the models you mentioned.

> If the past key values aren't needed then the inspection of the base model's forward shouldn't check for it.
Could you point us to the place of the `inspection` you mean here? 




See [https://github.com/huggingface/peft/blob/dbd40d96a15d9b8b04c3582bb9ea00ae24f56348/src/peft/peft_model.py#L828](https://github.com/huggingface/peft/blob/dbd40d96a15d9b8b04c3582bb9ea00ae24f56348/src/peft/peft_model.py#L828).

Also, the forward arguments for RobertaSelfAttention in the non-causal decoder case ("generate") [here](https://github.com/huggingface/transformers/blob/75a33d60f25d99ff8cdd657d6ba685dc4336a0d1/src/transformers/models/roberta/modeling_roberta.py#L213C13-L213C13). Maybe it's the case that PEFT shouldn't be using this in the first place even if it's useful for causal models.

This is indeed something to be addressed on PEFT side, as it uses
```
signature(self.base_model.forward)
```
by looking the `base_model`, which can take more arguments (as there are different types of models with head on top of the base model).