[seq2seq] translation tpu example doesnt work
Author: Stupack
Pull request: no

Hi im trying to run the `train_distil_marian_enro_tpu.sh` example in collab/kaggle tpus and for some reason it gives me the following output:
@sshleifer 
```
Exception in device=TPU:0: Cannot access data pointer of Tensor that doesn't have storage
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 330, in _mp_start_fn
    _start_fn(index, pf_cfg, fn, args)
  File "/usr/local/lib/python3.6/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 324, in _start_fn
    fn(gindex, *args)
  File "/content/transformers/examples/seq2seq/finetune_trainer.py", line 300, in _mp_fn
    main()
  File "/content/transformers/examples/seq2seq/finetune_trainer.py", line 249, in main
    model_path=model_args.model_name_or_path if os.path.isdir(model_args.model_name_or_path) else None
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 776, in train
    tr_loss += self.training_step(model, inputs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 1128, in training_step
    loss.backward()
  File "/usr/local/lib/python3.6/dist-packages/torch/tensor.py", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/usr/local/lib/python3.6/dist-packages/torch/autograd/__init__.py", line 132, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: Cannot access data pointer of Tensor that doesn't have storage
  0% 0/7158 [00:44<?, ?it/s]
Traceback (most recent call last):
  File "xla_spawn.py", line 72, in <module>
    main()
  File "xla_spawn.py", line 68, in main
    xmp.spawn(mod._mp_fn, args=(), nprocs=args.num_cores)
  File "/usr/local/lib/python3.6/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 395, in spawn
    start_method=start_method)
  File "/usr/local/lib/python3.6/dist-packages/torch/multiprocessing/spawn.py", line 157, in start_processes
    while not context.join():
  File "/usr/local/lib/python3.6/dist-packages/torch/multiprocessing/spawn.py", line 112, in join
    (error_index, exitcode)
Exception: process 0 terminated with exit code 

```
Related to this issue  #https://github.com/pytorch/xla/issues/929

Not sure how to solve it.  Thanks!

Comments:
Hi,
I'm experiencing the same issue while fine-tuning on TPU for marianMT. Code works well on GPU. On TPU, it throws similar exception: 
```python
RuntimeError: Cannot access data pointer of Tensor that doesn't have storage
```
Thanks!

Hi @chris-tng ,

could you post your env info, and the script/command that you are running ?

Sure, I'm using 
```shell
transformers                  4.0.1 
torch                         1.7.0+cu101    
torch-xla                     1.7            
torchsummary                  1.5.1          
torchtext                     0.3.1          
torchvision                   0.8.1+cu101
```
- command

```shell
!python xla_spawn.py --num_cores 8 finetune_trainer.py \
    --tokenizer_name "Helsinki-NLP/opus-mt-es-en" \
    --model_name_or_path "Helsinki-NLP/opus-mt-es-en" \
    --data_dir "/content/data" \
    --output_dir "/content/marian_es_en" --overwrite_output_dir \
    --learning_rate=3e-4 \
    --warmup_steps 500 \
    --per_device_train_batch_size=256 --per_device_eval_batch_size=256 \
    --freeze_encoder --freeze_embeds \
    --num_train_epochs=6 \
    --save_steps 3000 --eval_steps 3000 \
    --logging_first_step --logging_steps 200 \
    --max_source_length 128 \
    --max_target_length 128 --val_max_target_length 128 --test_max_target_length 128 \
    --do_train --do_eval --do_predict \
    --n_val 5000 --n_test 10000 --evaluation_strategy steps \
    --prediction_loss_only \
    --task translation --label_smoothing 0.1 \
    "$@"
```

Here is the error
```
[INFO|trainer.py:666] 2020-12-11 05:56:11,550 >>   Total train batch size (w. parallel, distributed & accumulation) = 2048
[INFO|trainer.py:667] 2020-12-11 05:56:11,550 >>   Gradient Accumulation steps = 1
[INFO|trainer.py:668] 2020-12-11 05:56:11,550 >>   Total optimization steps = 1830
  0% 0/1830 [00:00<?, ?it/s]Exception in device=TPU:0: Cannot access data pointer of Tensor that doesn't have storage
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 330, in _mp_start_fn
    _start_fn(index, pf_cfg, fn, args)
  File "/usr/local/lib/python3.6/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 324, in _start_fn
    fn(gindex, *args)
  File "/content/transformers/examples/seq2seq/finetune_trainer.py", line 309, in _mp_fn
    main()
  File "/content/transformers/examples/seq2seq/finetune_trainer.py", line 258, in main
    model_path=model_args.model_name_or_path if os.path.isdir(model_args.model_name_or_path) else None
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 747, in train
    tr_loss += self.training_step(model, inputs)
  File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 1089, in training_step
    loss.backward()
  File "/usr/local/lib/python3.6/dist-packages/torch/tensor.py", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/usr/local/lib/python3.6/dist-packages/torch/autograd/__init__.py", line 132, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: Cannot access data pointer of Tensor that doesn't have storage
  0% 0/1830 [03:34<?, ?it/s]
Traceback (most recent call last):
  File "xla_spawn.py", line 72, in <module>
    main()
  File "xla_spawn.py", line 68, in main
    xmp.spawn(mod._mp_fn, args=(), nprocs=args.num_cores)
  File "/usr/local/lib/python3.6/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 395, in spawn
    start_method=start_method)
  File "/usr/local/lib/python3.6/dist-packages/torch/multiprocessing/spawn.py", line 157, in start_processes
    while not context.join():
  File "/usr/local/lib/python3.6/dist-packages/torch/multiprocessing/spawn.py", line 112, in join
    (error_index, exitcode)
Exception: process 0 terminated with exit code 17
```


Hi @patil-suraj , any idea why it happens? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.