remove the extra `accelerator.prepare` 
Author: pacman100
Pull request: yes

# What does this PR do?
 Remove the extra `accelerator.prepare` that slipped in with multiple update from main 😅
1.  Fixes #23905

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I still get the same error @younesbelkada I have reinstalled from source but I still get the error. I'm in the latest commit `transformers @ git+https://github.com/huggingface/transformers.git@fabe17a726bbf6081cfbcc975d8ac451a81f3e2d` and you can tell from the stacktrace that the line numbers are different (due to the changes merged here https://github.com/huggingface/transformers/pull/23914/files)

```╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <cell line: 17>:17                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:1661 in train                    │
│                                                                                                  │
│   1658 │   │   inner_training_loop = find_executable_batch_size(                                 │
│   1659 │   │   │   self._inner_training_loop, self._train_batch_size, args.auto_find_batch_size  │
│   1660 │   │   )                                                                                 │
│ ❱ 1661 │   │   return inner_training_loop(                                                       │
│   1662 │   │   │   args=args,                                                                    │
│   1663 │   │   │   resume_from_checkpoint=resume_from_checkpoint,                                │
│   1664 │   │   │   trial=trial,                                                                  │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:1995 in _inner_training_loop     │
│                                                                                                  │
│   1992 │   │   │   │   │   │   │   │   args.max_grad_norm,                                       │
│   1993 │   │   │   │   │   │   │   )                                                             │
│   1994 │   │   │   │   │   │   else:                                                             │
│ ❱ 1995 │   │   │   │   │   │   │   self.accelerator.clip_grad_norm_(                             │
│   1996 │   │   │   │   │   │   │   │   model.parameters(),                                       │
│   1997 │   │   │   │   │   │   │   │   args.max_grad_norm,                                       │
│   1998 │   │   │   │   │   │   │   )                                                             │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py:1817 in clip_grad_norm_        │
│                                                                                                  │
│   1814 │   │   │   # `accelerator.backward(loss)` is doing that automatically. Therefore, its i  │
│   1815 │   │   │   # We cannot return the gradient norm because DeepSpeed does it.               │
│   1816 │   │   │   return None                                                                   │
│ ❱ 1817 │   │   self.unscale_gradients()                                                          │
│   1818 │   │   return torch.nn.utils.clip_grad_norm_(parameters, max_norm, norm_type=norm_type)  │
│   1819 │                                                                                         │
│   1820 │   def clip_grad_value_(self, parameters, clip_value):                                   │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py:1780 in unscale_gradients      │
│                                                                                                  │
│   1777 │   │   │   for opt in optimizer:                                                         │
│   1778 │   │   │   │   while isinstance(opt, AcceleratedOptimizer):                              │
│   1779 │   │   │   │   │   opt = opt.optimizer                                                   │
│ ❱ 1780 │   │   │   │   self.scaler.unscale_(opt)                                                 │
│   1781 │                                                                                         │
│   1782 │   def clip_grad_norm_(self, parameters, max_norm, norm_type=2):                         │
│   1783 │   │   """                                                                               │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/torch/cuda/amp/grad_scaler.py:275 in unscale_            │
│                                                                                                  │
│   272 │   │   optimizer_state = self._per_optimizer_states[id(optimizer)]                        │
│   273 │   │                                                                                      │
│   274 │   │   if optimizer_state["stage"] is OptState.UNSCALED:                                  │
│ ❱ 275 │   │   │   raise RuntimeError("unscale_() has already been called on this optimizer sin   │
│   276 │   │   elif optimizer_state["stage"] is OptState.STEPPED:                                 │
│   277 │   │   │   raise RuntimeError("unscale_() is being called after step().")                 │
│   278                                                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: unscale_() has already been called on this optimizer since the last update().
```

am I missing something? I'm just running this colab demo from a live event yesterday https://colab.research.google.com/drive/1ARmlaZZaKyAg6HTi57psFLPeh0hDRcPX?usp=sharing#scrollTo=Duak7T_B3VpJ

please check: https://github.com/huggingface/transformers/issues/23935#issuecomment-1571989596

Hi may I know what is the correct way to use accelerate in Trainer now? right now I manually do accelerate.prepare after initializing trainer by using `trainer.model_wrapped, trainer.optimizer, trainer.lr_scheduler = accelerator.prepare(trainer.model_wrapped, trainer.optimizer, trainer.lr_scheduler)`. Is that not needed at all?

> Hi may I know what is the correct way to use accelerate in Trainer now? right now I manually do accelerate.prepare after initializing trainer by using trainer.model_wrapped, trainer.optimizer, trainer.lr_scheduler = accelerator.prepare(trainer.model_wrapped, trainer.optimizer, trainer.lr_scheduler). Is that not needed at all?

Not needed now as it is being done internally in Trainer. It will prepare based on either the Trainer arguments or the accelerate config


Which argument should I use for that? Last time i checked there is nothing related

any help? I just checked doc and there is still nothing accelerate related

Hello @aliencaocao, you can use the Trainer as it was used previously and expect no changes from user's point of view. You can now use accelerate launcher directly too. Here are the relevant docs: https://huggingface.co/docs/transformers/main/en/main_classes/trainer#using-accelerate-launcher-with-trainer