ValueError raised during accelerate decoding 
Author: petroskarypis
Pull request: no

### System Info

transformers-4.28.1
python-3.11.3

### Who can help?

@gant

Example code found in this blogpost raises errors: [https://huggingface.co/blog/assisted-generation](https://huggingface.co/blog/assisted-generation).

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

prompt = "Alice and Bob"
checkpoint = "EleutherAI/pythia-1.4b-deduped"
assistant_checkpoint = "EleutherAI/pythia-160m-deduped"
device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
inputs = tokenizer(prompt, return_tensors="pt").to(device)

model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)
assistant_model = AutoModelForCausalLM.from_pretrained(assistant_checkpoint).to(device)
outputs = model.generate(**inputs, assistant_model=assistant_model)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))
['Alice and Bob are sitting in a bar. Alice is drinking a beer and Bob is drinking a']

ValueError                                Traceback (most recent call last)
Cell In[9], line 14
      1 # from transformers import AutoModelForCausalLM, AutoTokenizer
      2 # import torch
      3 
   (...)
     12 # model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)
     13 # assistant_model = AutoModelForCausalLM.from_pretrained(assistant_checkpoint).to(device)
---> 14 outputs = model.generate(**inputs, assistant_model=assistant_model)

File ~/anaconda3/envs/fcm-ape/lib/python3.11/site-packages/torch/utils/_contextlib.py:115, in context_decorator.<locals>.decorate_context(*args, **kwargs)
    112 @functools.wraps(func)
    113 def decorate_context(*args, **kwargs):
    114     with ctx_factory():
--> 115         return func(*args, **kwargs)

File ~/anaconda3/envs/fcm-ape/lib/python3.11/site-packages/transformers/generation/utils.py:1231, in GenerationMixin.generate(self, inputs, generation_config, logits_processor, stopping_criteria, prefix_allowed_tokens_fn, synced_gpus, streamer, **kwargs)
   1229 model_kwargs = generation_config.update(**kwargs)  # All unused kwargs must be model kwargs
   1230 generation_config.validate()
-> 1231 self._validate_model_kwargs(model_kwargs.copy())
   1233 # 2. Set generation parameters if not already defined
   1234 logits_processor = logits_processor if logits_processor is not None else LogitsProcessorList()

File ~/anaconda3/envs/fcm-ape/lib/python3.11/site-packages/transformers/generation/utils.py:1109, in GenerationMixin._validate_model_kwargs(self, model_kwargs)
...
   1110         f"The following `model_kwargs` are not used by the model: {unused_model_args} (note: typos in the"
   1111         " generate arguments will also show up in this list)"
   1112     )

ValueError: The following `model_kwargs` are not used by the model: ['assistant_model'] (note: typos in the generate arguments will also show up in this list)


### Expected behavior

['Alice and Bob are sitting in a bar. Alice is drinking a beer and Bob is drinking a']

Comments: