[seq2seq Example] Convert tensor to List[int] for decoding
Author: setu4993
Pull request: yes

Ran into an error today while using `finetune.py` where decoding kept failing because the output was a `torch.Tensor` instead of a `List[int]`. This fixes that.

Comments:
what are your versions? This doesn't break for me.
Try running 
```
transformers-cli env
```

The latest version from PyPI, v3.0.2. It doesn't happen while running locally but does break on remote executions, though.

Looking through the documentation, it made sense to me that an error would pop up since [`.generate()`](https://huggingface.co/transformers/model_doc/bart.html#transformers.BartForConditionalGeneration.generate) outputs a `torch.Tensor`, but [`tokenizer.decode()`](https://huggingface.co/transformers/main_classes/tokenizer.html#transformers.PreTrainedTokenizer.decode) (and batch_decode) expect `List[int]`

PyTorch 1.6.0 and Python 3.6, if that helps.

Can I see your traceback?

You can loop over tensors just like lists.

I have a custom `summarization_trainer.py` in there that calls the `main` from `finetune.py`.

Traceback:
```
File "summarization_trainer.py", line 157, in main
    return finetune_main(args, model)
  File "/opt/ml/code/finetune.py", line 460, in main
    logger=logger,
  File "/opt/ml/code/lightning_base.py", line 448, in generic_train
    trainer.fit(model)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 1044, in fit
    results = self.run_pretrain_routine(model)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 1196, in run_pretrain_routine
    False)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/evaluation_loop.py", line 293, in _evaluate
    output = self.evaluation_forward(model, batch, batch_idx, dataloader_idx, test_mode)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/evaluation_loop.py", line 470, in evaluation_forward
    output = model.validation_step(*args)
  File "/opt/ml/code/finetune.py", line 211, in validation_step
    return self._generative_step(batch)
  File "/opt/ml/code/finetune.py", line 255, in _generative_step
    preds: List[str] = self.ids_to_clean_text(generated_ids)
  File "/opt/ml/code/finetune.py", line 153, in ids_to_clean_text
    generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True
  File "/opt/conda/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 2254, in batch_decode
    return [self.decode(seq, **kwargs) for seq in sequences]
  File "/opt/conda/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 2254, in <listcomp>
    return [self.decode(seq, **kwargs) for seq in sequences]
  File "/opt/conda/lib/python3.6/site-packages/transformers/tokenization_utils_fast.py", line 439, in decode
    text = self._tokenizer.decode(token_ids, skip_special_tokens=skip_special_tokens)
  File "/opt/conda/lib/python3.6/site-packages/tokenizers/implementations/base_tokenizer.py", line 267, in decode
    return self._tokenizer.decode(ids, skip_special_tokens=skip_special_tokens)
TypeError
```

is there anything after TypeError?
I guess this is a torch 1.6 issue.

No, it doesn't say anything after `TypeError`. Could be a 1.6 issue, let me try to downgrade and report back.

Hmm, something's off. Now I'm seeing `TypeError` when trying to save the tokenizer during a checkpoint.
```
Traceback (most recent call last):
  File "/root/ds-sandbox/projects/abstractive_summarization/finetuning-bart/finetuning_bart/summarization_trainer.py", line 214, in <module>
    _ = slack_wrapper(args)
  File "/opt/conda/lib/python3.6/site-packages/knockknock/slack_sender.py", line 105, in wrapper_sender
    raise ex
  File "/opt/conda/lib/python3.6/site-packages/knockknock/slack_sender.py", line 63, in wrapper_sender
    value = func(*args, **kwargs)
  File "/root/ds-sandbox/projects/abstractive_summarization/finetuning-bart/finetuning_bart/summarization_trainer.py", line 159, in main
    return finetune_main(args, model)
  File "/root/ds-sandbox/projects/abstractive_summarization/finetuning-bart/finetuning_bart/finetune.py", line 463, in main
    logger=logger,
  File "/root/ds-sandbox/projects/abstractive_summarization/finetuning-bart/finetuning_bart/lightning_base.py", line 448, in generic_train
    trainer.fit(model)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 1003, in fit
    results = self.single_gpu_train(model)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/distrib_parts.py", line 186, in single_gpu_train
    results = self.run_pretrain_routine(model)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 1213, in run_pretrain_routine
    self.train()
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/training_loop.py", line 370, in train
    self.run_training_epoch()
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/training_loop.py", line 470, in run_training_epoch
    self.run_evaluation(test_mode=False)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/evaluation_loop.py", line 430, in run_evaluation
    self.on_validation_end()
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/callback_hook.py", line 112, in on_validation_end
    callback.on_validation_end(self, self.get_model())
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/utilities/distributed.py", line 12, in wrapped_fn
    return fn(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/callbacks/model_checkpoint.py", line 309, in on_validation_end
    self._do_check_save(filepath, current, epoch)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/callbacks/model_checkpoint.py", line 346, in _do_check_save
    self._save_model(filepath)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/callbacks/model_checkpoint.py", line 168, in _save_model
    self.save_function(filepath, self.save_weights_owandb: Program failed with code 1. Press ctrl-c to abort syncing.
nly)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/training_io.py", line 268, in save_checkpoint
    checkpoint = self.dump_checkpoint(weights_only)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/training_io.py", line 379, in dump_checkpoint
    model.on_save_checkpoint(checkpoint)
  File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/utilities/distributed.py", line 12, in wrapped_fn
    return fn(*args, **kwargs)
  File "/root/ds-sandbox/projects/abstractive_summarization/finetuning-bart/finetuning_bart/lightning_base.py", line 223, in on_save_checkpoint
    self.tokenizer.save_pretrained(save_path)
  File "/opt/conda/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 1379, in save_pretrained
    vocab_files = self.save_vocabulary(save_directory)
  File "/opt/conda/lib/python3.6/site-packages/transformers/tokenization_utils_fast.py", line 449, in save_vocabulary
    files = self._tokenizer.save_model(save_directory)
  File "/opt/conda/lib/python3.6/site-packages/tokenizers/implementations/base_tokenizer.py", line 323, in save_model
    return self._tokenizer.model.save(directory, name=name)
TypeError
```

I have the changes proposed in this PR in my code so it is clearly not the same.

^ is occurring on torch 1.5.1 and 1.6.0 both.

Update: I was creating a Fast tokenizer and passing it in during the creation of `SummarizationModule`. Switching to the one automatically created by the `BaseTransformer` avoids the last error.

The change in the PR is also related to the above error and creating a new tokenizer instead of an automatic initialization. Letting `SummarizationModule` and `BaseTransformer` deal with the creation of tokenizer instead doesn't raise those errors, however, that also means I can't use the Rust-based tokenizers.

Great catch! Would you mind making a new issue with a broken snippet that doesn't use PL? and tag @sshleifer 
E.g.
```
from transformers import BartTokenizerFast, BartForConditionalGeneration
...

```
?
Then we can try to fix the bug on the proper level of abstraction.

Thanks! Yes, agree that the right place for this is an issue. I'll try to create an issue that doesn't use PL later in the day.