ModuleAttributeError: 'GPT2LMHeadModel' object has no attribute 'backward'
Author: Octopirate1
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.2.1
- Platform: Linux-4.19.0-12-cloud-amd64-x86_64-with-debian-10.6
- Python version: 3.7.8
- PyTorch version (GPU?): 1.6.0a0+9907a3e (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No(?)

### Who can help

 albert, bert, GPT2, XLM: @LysandreJik
 Trainer: @sgugger

## To reproduce

Steps to reproduce the behavior:

1. Set up a TrainingArguments for a GPT2LMHeadModel with the following deepspeed config:

```
{
    "fp16": {
        "enabled": true,
        "loss_scale": 0,
        "loss_scale_window": 1000,
        "hysteresis": 2,
        "min_loss_scale": 1
    },

    "zero_optimization": {
        "stage": 2,
       "allgather_partitions": true,
       "allgather_bucket_size": 2e8,
       "reduce_scatter": true,
       "reduce_bucket_size": 2e8,
        "overlap_comm": true,
        "contiguous_gradients": true,
        "cpu_offload": false
    },

    "optimizer": {
        "type": "Adam",
        "params": {
            "adam_w_mode": true,
            "lr": 3e-5,
            "betas": [ 0.9, 0.999 ],
            "eps": 1e-8,
            "weight_decay": 3e-7
        }
    },

    "scheduler": {
        "type": "WarmupLR",
        "params": {
            "warmup_min_lr": 0,
            "warmup_max_lr": 3e-5,
            "warmup_num_steps": 500
        }
    }
}

```
2. Attempt to call `trainer.train()`.

## Expected behavior

Training should begin as expected.

## Believed bug location

It would appear that [line 1286 in trainer.py](https://github.com/huggingface/transformers/blob/76f36e183a825b8e5576256f4e057869b2e2df29/src/transformers/trainer.py#L1286) actually calls the `backward` method on the *model*, not the loss object. I will try rebuilding after fixing that line and seeing if it helps.


Comments:
> It would appear that line 1286 in trainer.py actually calls the backward method on the model, not the loss object. I will try rebuilding after fixing that line and seeing if it helps.

This is incorrect. It appears that the ``model_wrapped.module`` in the aforementioned trainer.py actually resolves to GPT2LMHeadModel. Another big shot in the dark, but maybe ``model_wrapped`` is never actually wrapping because I'm only using one GPU? It's very late where I live, I'll take another shot at this in the morning.

You then need to launch your script with the `deepspeed` launcher. Could you tell us which command you ran?
Also cc @stas00 since he added deepspeed to Trainer.

Yes, please tag me on any deepspeed issues.

Thank you for this report.

I think it's a bug, it should be:

```
self.deepspeed.backward(loss)
```

I will test and send a fix.


The merged PR closed this report, but should you still have an issue please don't hesitate to re-open it. 


Hello,
Thanks for this great deepspeed feature. I am also running into the same error both for 
DistilBertForSequenceClassification' object has no attribute 'backward'
and for
BertForSequenceClassification object has no attribute 'backward'

here is the full error:

> ---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-6-beaae64139c1> in <module>
     23 )
     24 
---> 25 trainer.train()

~/anaconda3/lib/python3.7/site-packages/transformers/trainer.py in train(self, model_path, trial)
    886                         tr_loss += self.training_step(model, inputs)
    887                 else:
--> 888                     tr_loss += self.training_step(model, inputs)
    889                 self._total_flos += self.floating_point_ops(inputs)
    890 

~/anaconda3/lib/python3.7/site-packages/transformers/trainer.py in training_step(self, model, inputs)
   1263         elif self.deepspeed:
   1264             # calling on DS engine (model_wrapped == DDP(Deepspeed(PretrainedModule)))
-> 1265             self.model_wrapped.module.backward(loss)
   1266         else:
   1267             loss.backward()

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in __getattr__(self, name)
    574                 return modules[name]
    575         raise AttributeError("'{}' object has no attribute '{}'".format(
--> 576             type(self).__name__, name))
    577 
    578     def __setattr__(self, name, value):

AttributeError: 'DistilBertForSequenceClassification' object has no attribute 'backward'

Any idea?
Thanks

@victorstorchan, can you please ensure you use an up-to-date master?

Thanks for your answer. I just pip installed transformers 1h ago. It should be up-to-date right?

no, it won't. pip installs the released version. you need the unreleased master build, which there are several ways to go about, one of them is just:

```
pip install git+https://github.com/huggingface/transformers
```





My bad! Thanks @stas00 

You did nothing wrong, @victorstorchan. 

I will propose an update to the installation page so that the distinction is loud and clear.