While running finetune.py in seq2seq examples on a custom dataset, I am getting the following error. 
Author: laibamehnaz
Pull request: no
Labels: name: wontfix

### This is what I'm getting:
Can I get some help? Thank you

Epoch 1:  34% 3410/10000 [1:43:10<3:19:22,  1.82s/it, loss=3.052, v_num=6]Traceback (most recent call last):
  File "finetune.py", line 344, in <module>
    main(args)
  File "finetune.py", line 322, in main
    logger=logger,
  File "/content/drive/My Drive/Colab Notebooks/transformers/examples/lightning_base.py", line 339, in generic_train
    trainer.fit(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/trainer.py", line 918, in fit
    self.single_gpu_train(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/distrib_parts.py", line 176, in single_gpu_train
    self.run_pretrain_routine(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/trainer.py", line 1093, in run_pretrain_routine
    self.train()
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py", line 375, in train
    self.run_training_epoch()
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py", line 458, in run_training_epoch
    _outputs = self.run_training_batch(batch, batch_idx)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py", line 634, in run_training_batch
    loss, batch_output = optimizer_closure()
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py", line 598, in optimizer_closure
    output_dict = self.training_forward(split_batch, batch_idx, opt_idx, self.hiddens)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/training_loop.py", line 773, in training_forward
    output = self.model.training_step(*args)
  File "finetune.py", line 131, in training_step
    loss_tensors = self._step(batch)
  File "finetune.py", line 126, in _step
    outputs = self(source_ids, attention_mask=source_mask, decoder_input_ids=y_ids, labels=lm_labels,)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "finetune.py", line 112, in forward
    return self.model(input_ids, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_t5.py", line 1175, in forward
    return_tuple=return_tuple,
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_t5.py", line 692, in forward
    input_ids = input_ids.view(-1, input_shape[-1])
RuntimeError: cannot reshape tensor of 0 elements into shape [-1, 0] because the unspecified dimension size -1 can be any value and is ambiguous
Epoch 1:  34%|███▍      | 3410/10000 [1:43:10<3:19:23,  1.82s/it, loss=3.052, ### v_num=6]



Comments:
Hi @laibamehnaz , what is your batch size ?

Here's my wild guess. I think one of your batch has empty lines. The batches are trimmed using the below function to remove excessive pad tokens

```python3
def trim_batch(
    input_ids, pad_token_id, attention_mask=None,
):
    """Remove columns that are populated exclusively by pad_token_id"""
    keep_column_mask = input_ids.ne(pad_token_id).any(dim=0)
    if attention_mask is None:
        return input_ids[:, keep_column_mask]
    else:
        return (input_ids[:, keep_column_mask], attention_mask[:, keep_column_mask])
```

if your batch has empty lines then after tokenizing it'll contain only pad tokens, and when its passed to trim batch it'll remove all those pad tokens and the batch will be empty i.e of shape [batch_size, 0]. This could the reason for the  `RuntimeError`

you can verify this with 
```python3
tok = T5Tokenizer.from_pretrained("t5-base")
text_batch = ["", ""]
enc = tok(text_batch, max_length=64, padding="max_length", return_tensors="pt")
print(enc['input_ids'].shape)
ids = trim_batch(enc['input_ids'], tok.pad_token_id)
input_shape = ids.size()
ids = ids.view(-1, input_shape[-1])
```

it'll give the same `RuntimeError`.

So I think you should check your dataset for empty lines, that should solve this issue 

Hi @patil-suraj ,
Thank you so much for your quick response. I have checked my dataset already for the same issue, and it didn't show any empty lines, and that's why I was so confused. But sure, I will check again. Also, my batch_size=1. 

If your bs is 1 then most probably blank line is the reason  

@laibamehnaz  can you just iterate through the dataloader with bs 1 and see when the shape comes out to be [bs, 0] this should allow you to pinpoint the error

Hi @patil-suraj, I tried what you suggested, but I do not get the same error. My smallest sentence is 1 word long, and there are no empty lines.  

Interesting, can you post the word here, maybe tokenizer is skipping it then. One more thing you can try is do forward pass only with that one word example  and see if it gives the error

Also pinging @sshleifer 

Also, there is [this line](https://github.com/huggingface/transformers/blob/ab0b3fa15c159034a55ee95ac39b7a6ba4527c4a/examples/seq2seq/finetune.py#L133) in finetune.py, which suggests that if your decoder_input_ids are length 0 and you're not adding special tokens (as was the case previously for t5), the labels would get truncated to shape (1, 0).

https://github.com/huggingface/transformers/blob/ab0b3fa15c159034a55ee95ac39b7a6ba4527c4a/examples/seq2seq/finetune.py#L133

Right, so one word example is encoded as single id and due to shifting and no special token, shape becomes (1,0). So one more reason to automatically add `
` with t5 tokenizer 

@laibamehnaz  so immediate solution for this issue is to add `
` at the end of every target example text

Alright, thank you so much. Will get back after trying it out. 

Works fine now. Thank you @patil-suraj.

hi @laibamehnaz , there's another bug in finetune.py when using T5. See issue  #5987 and PR #5994

Fix is easy, just check the change in the PR

Thanks a lot!

That PR is merged now, you can now just clone the master or pull if you've already cloned

I did notice that. Thanks a lot!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.