BART fine tuning on gpu issue
Author: Annu99
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

I am fine tuning BART by executing transformers/examples/seq2seq/finetune.sh as instructed.
I am getting the following error:

```
Traceback (most recent call last):
  File "finetune.py", line 346, in <module>
    main(args)
  File "finetune.py", line 324, in main
    logger=logger,
  File "/data/t-angand/transformers/examples/lightning_base.py", line 330, in generic_train
    trainer.fit(model)
  File "/home/angand/.conda/envs/bart_env/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 928, in fit
    self.single_gpu_train(model)
  File "/home/angand/.conda/envs/bart_env/lib/python3.6/site-packages/pytorch_lightning/trainer/distrib_parts.py", line 183, in single_gpu_train
    self.run_pretrain_routine(model)
  File "/home/angand/.conda/envs/bart_env/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 1086, in run_pretrain_routine
    False)
  File "/home/angand/.conda/envs/bart_env/lib/python3.6/site-packages/pytorch_lightning/trainer/evaluation_loop.py", line 291, in _evaluate
    output = self.evaluation_forward(model, batch, batch_idx, dataloader_idx, test_mode)
  File "/home/angand/.conda/envs/bart_env/lib/python3.6/site-packages/pytorch_lightning/trainer/evaluation_loop.py", line 462, in evaluation_forward
    output = model.validation_step(*args)
  File "finetune.py", line 136, in validation_step
    return self._generative_step(batch)
  File "finetune.py", line 163, in _generative_step
    generated_ids = self.model.generate(input_ids=source_ids, attention_mask=source_mask, use_cache=True,)
  File "/home/angand/.conda/envs/bart_env/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)
  File "/data/t-angand/transformers/src/transformers/modeling_utils.py", line 1159, in generate
    encoder_outputs: tuple = encoder(input_ids, attention_mask=attention_mask)
  File "/home/angand/.conda/envs/bart_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/data/t-angand/transformers/src/transformers/modeling_bart.py", line 303, in forward
    inputs_embeds = self.embed_tokens(input_ids) * self.embed_scale
  File "/home/angand/.conda/envs/bart_env/lib/python3.6/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/angand/.conda/envs/bart_env/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 114, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "/home/angand/.conda/envs/bart_env/lib/python3.6/site-packages/torch/nn/functional.py", line 1724, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected object of device type cuda but got device type cpu for argument #3 'index' in call to _th_index_select
```

Thanks is advance.

Comments:
Have you checked if your setup is working properly and if 
`torch.cuda.is_available()`
returns True?

Yeah
It gives initial output as:
Some weights of the model checkpoint at facebook/bart-large were not used when initializing BartForConditionalGeneration: ['encoder.version', 'decoder.version']
- This IS expected if you are initializing BartForConditionalGeneration from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
- This IS NOT expected if you are initializing BartForConditionalGeneration from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of BartForConditionalGeneration were not initialized from the model checkpoint at facebook/bart-large and are newly initialized: ['final_logits_bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
CUDA_VISIBLE_DEVICES: [0]
Using APEX 16bit precision.
Selected optimization level O1:  Insert automatic casts around Pytorch functions and Tensor methods.

Defaults for this optimization level are:
enabled                : True
opt_level              : O1
cast_model_type        : None
patch_torch_functions  : True
keep_batchnorm_fp32    : None
master_weights         : None
loss_scale             : dynamic
Processing user overrides (additional kwargs that are not None)...
After processing overrides, optimization options are:
enabled                : True
opt_level              : O1
cast_model_type        : None
patch_torch_functions  : True
keep_batchnorm_fp32    : None
master_weights         : None
loss_scale             : dynamic
Warning:  multi_tensor_applier fused unscale kernel is unavailable, possibly because apex was installed without --cuda_ext --cpp_ext. Using Python fallback.  Original ImportError was: ModuleNotFoundError("No module named 'amp_C'",)
Validation sanity check: 0it [00:00, ?it/s]
Then fails.

What's the output of `transformers-cli env`?

Sorry I didnt get what you are asking for.


Can you run the command `transformers-cli env` in terminal and paste the output here in code format (using ```)

It says command not found.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.