AttributeError: 'QAModel' object has no attribute 'automatic_optimization'
Author: emilzilyaev
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.2
- Platform: google colab
- `pytorch-lightning` version (GPU?): 1.2.0

<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people. -->

####Models:
MT5ForConditionalGeneration ('google/mt5-base')

Library:

- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik


Model I am using (MT5ForConditionalGeneration ('google/mt5-base')):

The problem arises when using:
pl.LightningDataModule with T5ForConditionalGeneration
* [ ] my own modified scripts: (give details below)
```
class QAModel(pl.LightningDataModule):

  [def __init__(self):
    super().__init__()
    self.model = MT5ForConditionalGeneration.from_pretrained(MODEL_NAME, return_dict=True)

  def forward(self, input_ids, attention_mask, labels=None):
    output = self.model(
        input_ids=input_ids,
        attention_mask=attention_mask,
        labels=labels
    )

    return output.loss, output.logits
  
  def training_step(self, batch, batch_idx):
    input_ids = batch['input_ids']
    attention_mask = batch['attention_mask']
    labels = batch['labels']
    loss, outputs = self(input_ids, attention_mask, labels)
    self.log('train_loss', loss, prog_bar=True, logger=True)
    return loss
  
  def validation_step(self, batch, batch_idx):
    input_ids = batch['input_ids']
    attention_mask = batch['attention_mask']
    labels = batch['labels']
    loss, outputs = self(input_ids, attention_mask, labels)
    self.log('val_loss', loss, prog_bar=True, logger=True)
    return loss

  def test_step(self, batch, batch_idx):
    input_ids = batch['input_ids']
    attention_mask = batch['attention_mask']
    labels = batch['labels']
    loss, outputs = self(input_ids, attention_mask, labels)
    self.log('train_loss', loss, prog_bar=True, logger=True)
    return loss

  def configure_optimizers(self):
    print('done')
    return AdamW(self.parameters(), lr=0.0001)] 
model = QAModel()
trainer.fit(model, data_module)
```


That is my colab code
https://colab.research.google.com/drive/1wRYnuQhkO8UvE2CtsJ09dGHy4R_nVTPd?usp=sharing

Thanks a lot!
****

Comments:
Hello! This issue seems to be with PyTorch Lightning rather than with Transformers.

> Hello! This issue seems to be with PyTorch Lightning rather than with Transformers.

Well, I've tried to pass MT5ForConditionalGeneration directly to fit() function, but got following error:

ModuleAttributeError: 'MT5ForConditionalGeneration' object has no attribute 'automatic_optimization'

As you'll see in your error:

```
/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/connectors/model_connector.py in copy_trainer_model_properties(self, model)
```

this originates from a PyTorch-Lightning error. Transformers has no `automatic_optimization` parameter, our models are plain PyTorch models.

Thank you.

thank you a lot