How to use `model.generate` with custom model with additional parameters?
Author: yeounyi
Pull request: no

Hi, I'm working on custom `BartForConditionalGeneration` model and it has additional parameters as following.
parameter with `'pos_' `prefix is the parameter I added personally. 

```
class SynSemBartForConditionalGeneration(BartPretrainedModel):
    ...
    def forward(
            self,
            input_ids=None,
            attention_mask=None,
            decoder_input_ids=None,
            decoder_attention_mask=None,
            head_mask=None,
            decoder_head_mask=None,
            encoder_outputs=None,
            past_key_values=None,
            inputs_embeds=None,
            decoder_inputs_embeds=None,
            labels=None,
            use_cache=None,
            output_attentions=None,
            output_hidden_states=None,
            return_dict=None,

            pos_input_ids=None,
            pos_attention_mask=None,
            pos_head_mask=None,
            pos_inputs_embeds=None,
            pos_output_attentions=None,
            pos_output_hidden_states=None,
            pos_return_dict=None,
    ):
```

I trained this model successfully, thanks to huggingface Trainer.
Then I tried to run `model.generate(input_ids=input_ids, pos_input_ids=pos_input_ids)`
I thought `pos_input_ids` is **"Additional model specific kwargs"** and it would **be forwarded to the `forward` function** of my model. 
However, it keeps giving me `TypeError: forward() got an unexpected keyword argument 'pos_input_ids'`.

I would appreciate any help 😥


Comments:
Hi there,

Could you post the full stack trace ?

Also, does the `encoder`s forward has this argument? Because if you see here, the extra `kwargs` are passed to the encoder as well

https://github.com/huggingface/transformers/blob/09b0bcfea98eb9e863248fd29566b6d1caf9a0ea/src/transformers/generation_utils.py#L410-L414

Does the full stack trace mean full error message?
```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/yeoun/.local/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/yeoun/.local/lib/python3.6/site-packages/transformers/generation_utils.py", line 927, in generate
    model_kwargs = self._prepare_encoder_decoder_kwargs_for_generation(input_ids, model_kwargs)
  File "/home/yeoun/.local/lib/python3.6/site-packages/transformers/generation_utils.py", line 412, in _prepare_encoder_decoder_kwargs_for_generation
    model_kwargs["encoder_outputs"]: ModelOutput = encoder(input_ids, return_dict=True, **encoder_kwargs)
  File "/home/yeoun/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
TypeError: forward() got an unexpected keyword argument 'pos_input_ids'
```

`encoder`'s `forward` doesn't have this argument. I connected two separate encoders, all has the same parameters as the original Bart encoder, but in the `model`'s `forward` method I used 'pos_' prefix for the inputs of the second encoder to distinct them.

```
class SynSemBartModel(BartPretrainedModel):
   ...

    def forward(
            self,
            input_ids=None,
            attention_mask=None,
            decoder_input_ids=None,
            decoder_attention_mask=None,
            head_mask=None,
            decoder_head_mask=None,
            encoder_outputs=None,
            past_key_values=None,
            inputs_embeds=None,
            decoder_inputs_embeds=None,
            use_cache=None,
            output_attentions=None,
            output_hidden_states=None,
            return_dict=None,

            pos_input_ids=None,
            pos_attention_mask=None,
            pos_head_mask=None,
            pos_inputs_embeds=None,
            pos_output_attentions=None,
            pos_output_hidden_states=None,
            pos_return_dict=None,

            pos_encoder_outputs=None,
    ):

          ...

            encoder_outputs = self.encoder(
                input_ids=input_ids,
                attention_mask=attention_mask,
                head_mask=head_mask,
                inputs_embeds=inputs_embeds,
                output_attentions=output_attentions,
                output_hidden_states=output_hidden_states,
                return_dict=return_dict,
            )

          ...

            pos_encoder_outputs = self.pos_encoder(
                input_ids=pos_input_ids,
                attention_mask=pos_attention_mask,
                head_mask=pos_head_mask,
                inputs_embeds=pos_inputs_embeds,
                output_attentions=pos_output_attentions,
                output_hidden_states=pos_output_hidden_states,
                return_dict=pos_return_dict,
            )
```

 I added two `encoders`' outputs and then feed into the `decoder`. 
Should I change my model structure in order to use this function? or is there any alternatives?
Thanks



In this case, you could override the `_prepare_encoder_decoder_kwargs_for_generation` method
and get the necessary output from both your encoder, add them, and then return that as the `last_hidden_state` of the encoder. The output is an instance of `BaseModelOutputClass`.


Another option is to use the generation methods separately, where you compute the necessary `encoder_outputs` and then pass that to `beam_serach` or whatever method you are using. Please refer to [beam_search](https://huggingface.co/transformers/main_classes/model.html#transformers.generation_utils.GenerationMixin.beam_search) docstring to see an example

It works after I override `_prepare_encoder_decoder_kwargs_for_generation` and `prepare_inputs_for_generation` method. Thanks a lot!