Removed @torch.no_grad() and in-place operations in optimizers for backwards
Author: shirayu
Pull request: yes

# What does this PR do?


In #23417, two``@torch.no_grad()`` lines were added before 

```python
def step(self, closure: Callable = None):
```

in class `AdamW` and `Adafactor`.

However, I think this should not be because this causes errors in backward.

Other optimizers in Pytorch have ``@_use_grad_for_differentiable`` lines before `def step`.

```python
@_use_grad_for_differentiable
def step(self, closure=None):
```
- Examples
    - https://github.com/pytorch/pytorch/blob/430cb3e1600e0aca742105a2cdf4a01d901955dd/torch/optim/adam.py#L122-L123
    - https://github.com/pytorch/pytorch/blob/430cb3e1600e0aca742105a2cdf4a01d901955dd/torch/optim/adamw.py#L149-L150

I also replaced in-place operations into assignments for backwards.

## Question

Should the following line also be replaced?

https://github.com/huggingface/transformers/blob/6ce6d62b6f20040129ec9831e7c4f6576402ea42/src/transformers/optimization.py#L728


## Context

I faced this problem when using PyTorch Lightning 2.0.3 with `transformers.optimization.Adafactor` as an optimizer.
With 3cf01b206 (one previous commit), this error did not occur.

```txt
 File "/path/to/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py", line 225, in optimizer_step
    return self.precision_plugin.optimizer_step(optimizer, model=model, closure=closure, **kwargs)
  File "/path/to/lib/python3.10/site-packages/pytorch_lightning/plugins/precision/precision_plugin.py", line 114, in optimizer_step
    return optimizer.step(closure=closure, **kwargs)
  File "/path/to/lib/python3.10/site-packages/torch/optim/optimizer.py", line 280, in wrapper
    out = func(*args, **kwargs)
  File "/path/to/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/path/to/lib/python3.10/site-packages/transformers/optimization.py", line 649, in step
    loss = closure()
  File "/path/to/lib/python3.10/site-packages/pytorch_lightning/plugins/precision/precision_plugin.py", line 101, in _wrap_closure
    closure_result = closure()
  File "/path/to/lib/python3.10/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 140, in __call__
    self._result = self.closure(*args, **kwargs)
  File "/path/to/lib/python3.10/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 135, in closure
    self._backward_fn(step_output.closure_loss)
  File "/path/to/lib/python3.10/site-packages/pytorch_lightning/loops/optimization/automatic.py", line 232, in backward_fn
    call._call_strategy_hook(self.trainer, "backward", loss, optimizer)
  File "/path/to/lib/python3.10/site-packages/pytorch_lightning/trainer/call.py", line 287, in _call_strategy_hook
    output = fn(*args, **kwargs)
  File "/path/to/lib/python3.10/site-packages/pytorch_lightning/strategies/strategy.py", line 200, in backward
    self.precision_plugin.backward(closure_loss, self.lightning_module, optimizer, *args, **kwargs)
  File "/path/to/lib/python3.10/site-packages/pytorch_lightning/plugins/precision/precision_plugin.py", line 67, in backward
    model.backward(tensor, *args, **kwargs)
  File "/path/to/lib/python3.10/site-packages/pytorch_lightning/core/module.py", line 1046, in backward
    loss.backward(*args, **kwargs)
  File "/path/to/lib/python3.10/site-packages/torch/_tensor.py", line 487, in backward
    torch.autograd.backward(
  File "/path/to/lib/python3.10/site-packages/torch/autograd/__init__.py", line 200, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn
Epoch 0:   0%|          | 0/2852 [00:01<?, ?it/s]
```

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->

- PyTorch: @sgugger


Comments:
_The documentation is not available anymore as the PR was closed or merged._

I consider this pull request is insufficient because tests have failed.
I welcome comments on the fix!

Transformers is primarily a library of models, not optimizers. I would recommend not using the AdamW/Adafactor from the library (which are going to be removed in the next major version) and use another implementation :-)

Thank you for the comment!

All right, I found an implementation `fairseq.optim.adafactor.Adafactor` and I will use it.

https://github.com/pytorch/pytorch/issues/30446