TypeError in tensorflow/run_summarization.py
Author: bpraveenk
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.11.0.dev0
- Platform:  Linux-5.4.0-1055-azure-x86_64-with-glibc2.10
- Python version: 3.8.1
- PyTorch version (GPU?):
- Tensorflow version (GPU?): 2.5.0 (Yes)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Distributed

### Who can help:

@patrickvonplaten, @patil-suraj, @Rocketknight

Models: facebook/bart 
datasets: xsum

* [ ] the official example scripts: (give details below)
run_summarization.py
relative path: examples/tensorflow/summarization

Steps to reproduce the behavior: (Note that --max_train_samples is optional)
python run_summarization.py  --model_name_or_path facebook/bart-base --dataset_name xsum --dataset_config "3.0.0" --output_dir /tmp/tst-summarization  --per_device_train_batch_size 4 --per_device_eval_batch_size 4 --num_train_epochs 3 --do_train --do_eval --max_train_samples 100

Error message
- INFO - __main__ - Evaluation...
  0%|                                                                                                                                                    | 0/2833 [00:01<?, ?it/s]
Traceback (most recent call last):
  File "run_summarization.py", line 663, in <module>
    main()
  File "run_summarization.py", line 639, in main
    generated_tokens = model.generate(**batch)
  File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/pbodigut1/code/Users/pbodigut/transformers/v-4.10/transformers/src/transformers/generation_tf_utils.py", line 736, in generate
    output = self._generate_beam_search(
  File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/pbodigut1/code/Users/pbodigut/transformers/v-4.10/transformers/src/transformers/generation_tf_utils.py", line 1102, in _generate_beam_search
    model_inputs = self.prepare_inputs_for_generation(
TypeError: prepare_inputs_for_generation() got multiple values for argument 'decoder_input_ids'

## Expected behavior

Successfully run the evaluation step.


Comments:
Hello, 
Facing the same issue while using https://github.com/huggingface/transformers/tree/master/examples/tensorflow/translation for finetuning MarianMT's opus-mt-en-hi model. 


## Environment info: 
transformers = '4.12.0.dev0'
tensorflow = 2.6.0
Python version: 3.8.5
Using GPU in script?: Yes

## Stack Trace: 

Traceback (most recent call last):
  File "run_translation.py", line 626, in <module>
    main()
  File "run_translation.py", line 610, in main
    generated_tokens = model.generate( input_ids = input_ids, attention_mask = attention_mask, decoder_input_ids = decoder_input_ids, labels = labels)
  File "/home/t-hdiddee/hf/lib/python3.8/site-packages/transformers/generation_tf_utils.py", line 736, in generate
    output = self._generate_beam_search(
  File "/home/t-hdiddee/hf/lib/python3.8/site-packages/transformers/generation_tf_utils.py", line 1102, in _generate_beam_search
    model_inputs = self.prepare_inputs_for_generation(
TypeError: prepare_inputs_for_generation() got multiple values for argument 'decoder_input_ids'



@Rocketknight1, wondering if there is any update on this issue.

@Rocketknight1 - could you take a look? :-)

I tested this simple fix, and it seems to work. In _run_summarization.py_, I added the line to remove ```decoder_input_ids``` from the batch dictionary (see below). My guess is that duplicate copies of ```decoder_input_ids``` are being passed to `prepare_inputs_for_generation()` function, after invoking ```model.generate(**batch)```.  

```
if training_args.do_eval:
            logger.info("Evaluation...")
            for batch, labels in tqdm(
                tf_eval_dataset, total=len(eval_dataset) // training_args.per_device_eval_batch_size
            ):
             decoder_input_ids = batch.pop("decoder_input_ids", None)
```



@patrickvonplaten and @Rocketknight1 could you confirm if the change looks good to you and the fix doesn't break the BART model inference logic?





This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Hi @bpraveenk, firstly, sorry for the very long delay here! You're correct that supplying `decoder_input_ids` seems to break `generate()`. Your solution to `pop()` that key from the dictionary probably fixes the issue, but the underlying problem is that `generate()` simply doesn't know what to do with a `decoder_input_ids` input key. We're currently in the middle of entirely refactoring our TF `generate()` function for performance/bugfixing reasons, so I'll put that in the list of stuff to be investigated there, but I don't have a timeline on it unfortunately.

In the meantime, if you want to submit a PR with that change, please do! However, it will definitely only be a temporary workaround until we figure things out properly.

Thank you for confirming the fix @Rocketknight1 . Could you point me to the PR submission instructions, I will submit the fix and I will add a comment that it is a temporary fix. 

Hi @bpraveenk, thanks for the offer! We have instructions for contributors [here](https://huggingface.co/docs/transformers/contributing).

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.