Rag-end2end
Author: Rajdoshi99
Pull request: no

### System Info

```shell
raise misconfigurationexception( pytorch_lightning.utilities.exceptions.misconfigurationexception: the provided lr scheduler `lambdalr` doesn't follow pytorch's lrscheduler api. you should override the `lightningmodule.lr_scheduler_step` hook with your own logic if you are using a custom lr scheduler. stopped all 7 ray processes.

pytorch_lightning=1.6.4
```


### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

raise misconfigurationexception( pytorch_lightning.utilities.exceptions.misconfigurationexception: the provided lr scheduler `lambdalr` doesn't follow pytorch's lrscheduler api. you should override the `lightningmodule.lr_scheduler_step` hook with your own logic if you are using a custom lr scheduler. stopped all 7 ray processes.

### Expected behavior

```shell
Its not working, it was working previously but now there is some misconfigurations error.
```


### Checklist

- [X] I have read the migration guide in the readme. ([pytorch-transformers](https://github.com/huggingface/transformers#migrating-from-pytorch-transformers-to-transformers); [pytorch-pretrained-bert](https://github.com/huggingface/transformers#migrating-from-pytorch-pretrained-bert-to-transformers))
- [X] I checked if a related official extension example runs on my machine.

Comments:
Hi @Rajdoshi99, thanks for raising this issue!

It seems the issue is coming from pytorch lighnting. So that we can best help, could you give us more information about the error and how to reproduce. Specifically: 
* Your environment. Run `transformers-cli env` in the terminal to get the necessary info to share
* A snippet of code that we can run to try and reproduce the error
* A full trackback of the error that occurred 

Following from my comment above ^ - this is likely an issue with the pytorch lightning version and its compatibility with the example. 

Pytorch Lighting 1.6.4 was released last June, whereas this example is three years old. We don't actively maintain the examples in the library. I would recommend downgrading the pytorch lighting version in your environment if you wish to run it. 

HI @amyeroberts 
Traceback (most recent call last):
  File "/home/ec2-user/SageMaker/transformers/examples/research_projects/rag-end2end-retriever/finetune_rag.py", line 815, in <module>
    main(args)
  File "/home/ec2-user/SageMaker/transformers/examples/research_projects/rag-end2end-retriever/finetune_rag.py", line 780, in main
    trainer: pl.Trainer = generic_train(
  File "/home/ec2-user/SageMaker/transformers/examples/research_projects/rag-end2end-retriever/lightning_base.py", line 410, in generic_train
    trainer.fit(model)
  File "/home/ec2-user/SageMaker/.cs/conda/envs/codeserver_py39/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 770, in fit
    self._call_and_handle_interrupt(
  File "/home/ec2-user/SageMaker/.cs/conda/envs/codeserver_py39/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 723, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/home/ec2-user/SageMaker/.cs/conda/envs/codeserver_py39/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 811, in _fit_impl
    results = self._run(model, ckpt_path=self.ckpt_path)
  File "/home/ec2-user/SageMaker/.cs/conda/envs/codeserver_py39/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1217, in _run
    self.strategy.setup(self)
  File "/home/ec2-user/SageMaker/.cs/conda/envs/codeserver_py39/lib/python3.9/site-packages/pytorch_lightning/strategies/ddp.py", line 179, in setup
    self.setup_optimizers(trainer)
  File "/home/ec2-user/SageMaker/.cs/conda/envs/codeserver_py39/lib/python3.9/site-packages/pytorch_lightning/strategies/strategy.py", line 128, in setup_optimizers
    self.optimizers, self.lr_scheduler_configs, self.optimizer_frequencies = _init_optimizers_and_lr_schedulers(
  File "/home/ec2-user/SageMaker/.cs/conda/envs/codeserver_py39/lib/python3.9/site-packages/pytorch_lightning/core/optimizer.py", line 195, in _init_optimizers_and_lr_schedulers
    _validate_scheduler_api(lr_scheduler_configs, model)
  File "/home/ec2-user/SageMaker/.cs/conda/envs/codeserver_py39/lib/python3.9/site-packages/pytorch_lightning/core/optimizer.py", line 350, in _validate_scheduler_api
    raise MisconfigurationException(
pytorch_lightning.utilities.exceptions.MisconfigurationException: The provided lr scheduler `LambdaLR` doesn't follow PyTorch's LRScheduler API. You should override the `LightningModule.lr_scheduler_step` hook with your own logic if you are using a custom LR scheduler.

Rag-End2End Retriever




Transformer CLI ENV

- `transformers` version: 4.27.1
- Platform: Linux-5.10.157-139.675.amzn2.x86_64-x86_64-with-glibc2.26
- Python version: 3.9.16
- Huggingface_hub version: 0.13.2
- PyTorch version (GPU?): 2.0.0+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <yes>
- Using distributed or parallel set-up in script?: <ray>

Hi @Rajdoshi99, thanks for providing this information! 

Looking at the traceback, the issue is indeed arising from pytorch lightning itself and its compatibility with the script. We don't actively maintain the research examples. If you wish to run the script I would suggest downgrading the pytorch lighting version in your environment. As the script is old, I unfortunately can't guarantee that will be enough to make it work.  

@Rajdoshi99 Hey Raj, I'm getting the same error. I tried different versions of PyTorch Lightning with no success. Were you able to fix this bug?